Volume 19 • Issue 1 • PP: 35–46 • 2027
An Information-Theoretic Framework for Evaluating Class Imbalance and Rare-Attack Observability in IoT Security Benchmarks
Open Access & Copyright
© 2027 The Author(s). Published by ASPG. This article is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
View PDF Abstract
IoT intrusion-detection studies often treat a benchmark as adequate once it is large, recent, and widely used. That assumption can fail. This study audits five public IoT and IIoT security benchmarks—CICIoT2023, Edge-IIoTset, ToN-IoT, BoT-IoT, and IoT-23—from their published class populations and introduces no synthetic classifier results; the audit combines Shannon entropy, Hill diversity, Simpson concentration, Gini inequality, rank-frequency tail fitting, rare-class observation probability, Wilson uncertainty, and a constrained power-tempered sampling model. Across the five corpora, record counts range from 461,043 to 73,370,443, normalized entropy from 0.421 to 0.600, and effective Shannon diversity from 2.20 to 3.98 classes. Size is a poor proxy for label adequacy. IoT-23 is the clearest case: its 6.05 million records include a singleton class, so a random 1% sample has only about a 1%probability of observing that label, while the worst-case Wilson half-width for a class-conditioned proportion is 0.445. We then define qi(τ) = piτ /∑k pτk and prove that its entropy is non-increasing in τ; at a 1% training budget with a target of 25 expected observations in the rarest class, the minimum-flattening exponent is 1.000 for CICIoT2023, 0.855 for Edge-IIoTset, 0.807 for ToN-IoT, 0.954 for BoT-IoT, and 0.450 for IoT-23. The last case needs an approximately 2,500-fold uplift of the singleton label. Such an uplift changes training exposure to one observation; it cannot create independent evidence. The resulting audit protocol therefore separates training rebalancing from evaluation validity and gives quantitative checks that can be applied before IoT benchmark results are interpreted as class-wide security evidence.
Keywords
References
[1] R. Baazeem, “An ensemble boosting algorithm based intrusion detection system for smart internet of things environment,” Journal of Intelligent Systems and Inter-net of Things, vol. 13, no. 2, pp. 272–292, 2024.
[2] P. J. Assudani, N. V. S. Pavan Kumar, K. Mohanambal, and R. Chitra, “Explainable artificial intelligence driven intrusion detection system for enhancing reliability and interpretability in IoT based network security solutions,” Journal of Intelligent Systems and Internet of Things, vol. 17, no. 1, pp. 219–238, 2025.
[3] A. Sleem, “Intelligent and secure detection of cyber-attacks in industrial internet of things: A federated learning framework,” Journal of Intelligent Systems and Internet of Things, vol. 7, no. 1, pp. 51–61, 2022.
[4] W. A. H. Salman, “A distributed intrusion detection using long short-term memory-gradient repeating unit and enhanced density peak clustering for real-time cyber threat detection,” Journal of Intelligent Systems and Internet of Things, vol. 18, no. 1, pp. 140–149, 2026.
[5] R. Abdulhammed, S. M. M. Najeeb, R. R. Ali, and M. A. Jubair, “Securing IoT through intrusion detection systems: An overview,” Journal of Intelligent Systems and Internet of Things, vol. 15, no. 2, pp. 183–190, 2025.
[6] O. Abualghanam, M. Al-Essa, W. Almobaideen, M. Qatawneh, and A. S. Al-Shamayleh, “Proactive DoS and DDoS attack detection through behavior-based threat intelligence,” Electronics, vol. 15, no. 12, p. 2559, 2026.
[7] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Transactions on Knowledge and Data En-gineering, vol. 21, no. 9, pp. 1263–1284, 2009.
[8] T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLOS ONE, vol. 10, no. 3, p. e0118432, 2015.
[9] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002.
[10] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie, “Class-balanced loss based on effective number of sam-ples,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9268–9277, 2019.
[11] A. K. Menon, S. Jayasumana, A. S. Rawat, H. Jain, A. Veit, and S. Kumar, “Long-tail learning via logit adjustment,” in International Conference on Learning Representations, 2021.
[12] E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, “CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment,” Sensors, vol. 23, no. 13, p. 5941, 2023.
[13] M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras, and H. Janicke, “Edge-IIoTset: A new comprehensive real-istic cyber security dataset of IoT and IIoT applications for centralized and federated learning,” IEEE Access, vol. 10, pp. 40 281–40 306, 2022.
[14] A. A. Ahmed and T. A. A. Abdullah, “An enhanced XGBoost-based framework for efficient multi-class cy-ber threat detection in industrial IoT networks,” Tech-nologies, vol. 14, no. 5, p. 274, 2026.
[15] A. Alsaedi, N. Moustafa, Z. Tari, A. Mahmood, and A. Anwar, “TON_IoT telemetry dataset: A new gener-ation dataset of IoT and IIoT for data-driven intrusion detection systems,” IEEE Access, vol. 8, pp. 165 130–165 150, 2020.
[16] R. Elsayed, R. Hamada, M. Hammoudeh, M. Abdalla, and S. A. Elsaid, “A hierarchical deep learning-based intrusion detection architecture for clustered internet of things,” Journal of Sensor and Actuator Networks, vol. 12, no. 1, p. 3, 2023.
[17] N. Koroniotis, N. Moustafa, E. Sitnikova, and B. Turn-bull, “Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-IoT dataset,” Future Generation Com-puter Systems, vol. 100, pp. 779–796, 2019.
[18] S. Garcia, A. Parmisano, and M. J. Erquiaga, “IoT-23: A labeled dataset with malicious and benign IoT network traffic,” Zenodo, 2020.
[19] H. Ghani, S. Salekzamankhani, and B. Virdee, “Statis-tical and multivariate analysis of the IoT-23 dataset: A comprehensive approach to network traffic pattern dis-covery,” Journal of Cybersecurity and Privacy, vol. 5, no. 4, p. 112, 2025.
[20] C. E. Shannon, “A mathematical theory of communica-tion,” Bell System Technical Journal, vol. 27, pp. 379–423 and 623–656, 1948.
[21] M. O. Hill, “Diversity and evenness: A unifying nota-tion and its consequences,” Ecology, vol. 54, no. 2, pp. 427–432, 1973.
[22] E. H. Simpson, “Measurement of diversity,” Nature, vol. 163, p. 688, 1949.
[23] T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ: Wiley, 2006.
[24] E. B. Wilson, “Probable inference, the law of succes-sion, and statistical inference,” Journal of the American Statistical Association, vol. 22, no. 158, pp. 209–212, 1927.
Cite This Article
Choose your preferred format
Publisher's Note
The statements, opinions, and data presented in this article are solely those of the author(s) and do not necessarily represent those of ASPG, the journal, or its editors. ASPG and the editors disclaim responsibility for any harm arising from the use of any ideas, methods, instructions, or products described in this article, to the fullest extent permitted by applicable law.