AI-Based Anomaly Detection for Cloud Logs and Distributed Traces
DOI:
https://doi.org/10.63282/3050-922X.IJERET-V7I2P103Keywords:
Cloud Security, Kubernetes, Artificial Intelligence, Anomaly Detection, Log Analysis, Distributed TracingAbstract
Cloud-native infrastructures built on microservices, containers, and distributed tracing systems generate massive volumes of heterogeneous telemetry data, including logs, metrics, and traces. Traditional rule-based anomaly detection techniques are increasingly inadequate due to their rigidity, high false positive rates, and inability to adapt to dynamic workloads. This paper introduces a machine learning-based anomaly detection model of cloud logs and distributed traces that uses machine learning and deep learning frameworks to detect anomalous behavior in a system in real-time. The proposed system incorporates multi-modal telemetry over a scalable Kubernetes implementation and implements methods including Isolation Forest and Autoencoders to perform unsupervised anomaly detection. By fusing log and trace data, the framework captures both semantic and structural dependencies across distributed services. Experimental evaluation in a simulated cloud environment demonstrates strong performance, achieving precision of up to 0.92, recall of 0.95, and F1-score of 0.94 for container escape detection, while maintaining an AUC above 0.95 across scenarios. Resource abuse detection had a precision of 0.91 and recall of 0.93, and unauthorized API usage had an F1-score of 0.89. These outcomes demonstrate high accuracy, memory and strength over the baseline models, which is why the method can be used in contemporary cloud security applications. It also addresses the aspect of scaling and real-time deployment in the setting of a production environment.
References
[1] A. Gupta, “What Is The Right Security Posture? A Perspective on Cloud Computing Security Threats and Risk Assessment,” Int. J. Emerg. Res. Eng. Technol., vol. 4, no. 4, 2023, doi: 10.63282/3050-922X.IJERET-V4I4P112.
[2] S. Bhat, S. R. Sirikonda, V. Katoch, and R. Jain, “Carbon-Kube: A Kubernetes-Native Framework for Multi-Objective Carbon-Aware Scheduling of Big Data Pipelines,” in 2026 9th International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), IEEE, Feb. 2026, pp. 1–6. doi: 10.1109/IEMENTech202669403.2026.11434192.
[3] N. Radhasharan, “Real-Time Edge-To-Cloud Intelligence Architecture For Autonomous Drilling Systems,” J. Int. Cris. RISK Commun. Res., vol. 9, no. 1, 2026.
[4] G. Sarraf and V. Pal, “Autonomous Threat Detection and Response in Cloud Security: A Comprehensive Survey of AI-Driven Strategies,” Int. J. Emerg. Res. Eng. Technol., vol. 6, no. 4, 2025, doi: 10.63282/3050-922X.IJERET-V6I4P114.
[5] A. Parupalli and H. Kali, “An In-Depth Review of Cost Optimization Tactics in Multi-Cloud Frameworks,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 3, no. 5, pp. 1043–1052, Jun. 2023, doi: 10.48175/IJARSCT-11937Q.
[6] S. Singamsetty, “An Intelligent Framework for Secure and Fair Cloud Resource Distribution,” in 2025 7th International Conference on Innovative Data Communication Technologies and Application (ICIDCA), IEEE, Oct. 2025, pp. 686–690. doi: 10.1109/ICIDCA66325.2025.11280502.
[7] S. K. Chintagunta, “Enhancing Cloud Database Security Through Intelligent Threat Detection and Risk Mitigation,” TIJER – Int. Res. J., vol. 9, no. 10, pp. 49–55, 2022.
[8] C. Carrión, “Kubernetes as a standard container orchestrator-a bibliometric analysis,” J. Grid Comput., vol. 20, no. 4, p. 42, 2022.
[9] B. Singh, H. Singh, and T. Banerjee, “Strengthening Modern IAM Authentication with Quantum Cryptography and Anti-Phishing Techniques,” Sarcouncil J. Eng. Comput. Sci., vol. 4, no. 10, Oct. 2025, doi: 10.5281/zenodo.17260292.
[10] M. R. Konatham, D. P. Guda, K. Kaushik, W. Sarma, R. Sharma, and M. Soni, “Explainable Deep Learning Framework for Real-Time Threat Hunting and Anomaly Attribution in Enterprise Networks,” in 2025 2nd International Conference on Integration of Computational Intelligent System (ICICIS), IEEE, Sep. 2025, pp. 1–6. doi: 10.1109/ICICIS65613.2025.11371132.
[11] V. Sharma, “Cloud-Native 5G Deployments: Kubernetes and Microservices in Telco Networks,” Int. J. Innov. Res. Eng. Multidiscip. Phys. Sci., vol. 10, no. 3, pp. 1–8, May 2022, doi: 10.37082/IJIRMPS.v10.i3.232706.
[12] J. Sajja and N. Kolli, “Towards a Unified Framework for Enterprise Data Transformation: Cloud Architecture, Governance, and Intelligent Automation,” J. Inf. Syst. Eng. Manag., vol. 9, no. 4, p. 20, 2024.
[13] A. Warrier, “Securing and Scaling API Gateways in Hybrid Environments,” J. Artif. Intell. Mach. Learn. Data Sci., vol. 3, no. 3, pp. 2914–2920, Sep. 2025, doi: 10.51219/JAIMLD/Arjun-warrier/607.
[14] D. Bhattacharjee, “Design and Evaluation of Deep Generative AI Model for Intrusion Detection in Cyber Threat Monitoring,” in 2025 7th International Symposium on Advanced Electrical and Communication Technologies (ISAECT), IEEE, Dec. 2025, pp. 1–6. doi: 10.1109/ISAECT68904.2025.11318752.
[15] H. P. Cyril, “DeepNetDetect: A Deep Learning-Based Approach for Early Anomaly Detection in Network Traffic,” in 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC), IEEE, Feb. 2026, pp. 1–6. doi: 10.1109/ICAIC67076.2026.11395734.
[16] R. Dattangire, R. Vaidya, D. Biradar, and A. Joon, “Exploring the Tangible Impact of Artificial Intelligence and Machine Learning: Bridging the Gap between Hype and Reality,” in 2024 1st International Conference on Advanced Computing and Emerging Technologies (ACET), IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/ACET61898.2024.10730334.
[17] K. K. Mohammed, “The Future is Cloud: Modernizing Big Data for the Cloud Era,” Int. J. Sci. Res. Eng. Trends, vol. 11, no. 5, pp. 1–5, 2025.
[18] J. E. Kofi, “Data-Driven Cloud Workload Optimization Using Machine Learning Modeling for Proactive Resource Management,” Int. J. Emerg. Res. Eng. Technol., vol. 6, no. 4, pp. 27–37, 2025, doi: 10.63282/3050-922X.IJERET-V6I4P104.
[19] M. Parikh, A. A. Soni, S. M. Shah, and A. R. Jha, “Big Data Workload Profiling for Energy-Aware Cloud Resource Management.” 2026. doi: 10.48550/arXiv.2601.11935.
[20] E. Malul, Y. Meidan, D. Mimran, Y. Elovici, and A. Shabtai, “GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection, Localization, Reasoning, and Remediation,” May 2024, doi: 10.48550/arXiv.2405.19954.
[21] M. S. Islam, M. S. Rakha, W. Pourmajidi, J. Sivaloganathan, J. Steinbacher, and A. Miranskyy, “Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset,” Jan. 2025, doi: 10.1109/ICSE-SEIP66354.2025.00039.
[22] S. Potluri, “A Deep Learning-Driven Framework for Detecting Anomalous Data Breaches in Distributed Cloud Storage Infrastructures,” Int. J. Artif. Intell. Data Sci. Mach. Learn., vol. 5, no. 3, pp. 80–87, Oct. 2024, doi: 10.63282/3050-9262.IJAIDSML-V5I3P109.
[23] X. Wei et al., “Log‐based anomaly detection for distributed systems: State of the art, industry experience, and open issues,” J. Softw. Evol. Process, vol. 36, no. 8, Aug. 2024, doi: 10.1002/smr.2650.
[24] C. Zhang et al., “DeepTraLog: trace-log combined microservice anomaly detection through graph-based deep learning,” in Proceedings of the 44th International Conference on Software Engineering, New York, NY, USA: ACM, May 2022, pp. 623–634. doi: 10.1145/3510003.3510180.
[25] M. Almansoori and M. Telek, “Anomaly Detection using combination of Autoencoder and Isolation Forest,” in 1st Workshop on Intelligent Infocommunication Networks, Systems and Services, Online: Budapest University of Technology and Economics, Feb. 2023, pp. 25–30. doi: 10.3311/WINS2023-005.
[26] D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S.-K. Ng, “MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019, pp. 703–716. doi: 10.1007/978-3-030-30490-4_56.
[27] G. Mattera, R. Mattera, S. Vespoli, and E. Salatiello, “Anomaly detection in manufacturing systems with temporal networks and unsupervised machine learning,” Comput. Ind. Eng., vol. 203, p. 111023, May 2025, doi: 10.1016/j.cie.2025.111023.
[28] J. Nobre, E. J. S. Pires, and A. Reis, “Anomaly Detection in Microservice-Based Systems,” Appl. Sci., vol. 13, no. 13, 2023, doi: 10.3390/app13137891.
