ZeroDay Malware Identification in Cloud Virtual Machines Using Temporal Behavioral Features and Deep Ensemble Learning
Keywords:
Behavioral analysis, Intrusion detection, LSTM, Machine learning, Malware detection, Random Forest, Support Vector MachineAbstract
Cloud computing has become an integral part of the way the modern organization runs. Cloud platforms are essential for businesses to store sensitive information, run applications, and provide large-scale services. The more that people start using the cloud, the more the risk of malware increases. Attackers have created new tools that are tailored to the cloud; many of these tools are not detected by traditional pattern- or signature-based tools. This paper presents a complete machine-learning-based malware detection system, which detects both known and unknown malware in cloud environments by analyzing malware behavior. The system tracks running processes and not specific patterns in the software. It continually gathers information on system calls, CPU usage, memory consumption, file system access, and network connections for virtual machines running on a private OpenStack cloud. These behavioral signals are then transformed into feature vectors and fed into three machine learning models: Random Forest, Support Vector Machine, and a Long Short-Term Memory neural network. Finally, the models make a classification decision by majority voting. The system was developed and evaluated using a set of 26,500 malware samples from five malware classes. The ensemble accurately classifies 98.7% of the cases, with a precision of 98.9% and a recall of 98.5%, while producing a very low false positive rate of 1.2%. The system also proves to be very good at generating zero-day threats, being able to identify previously unknown malware families over 89% accurately. The average detection latency is less than 400 milliseconds, and the processing overhead is less than 3% per VM. The findings in this work were validated and could be considered a reliable, scalable and feasible method for securing modern cloud environments using behavioral analysis and ensemble machine learning.
References
N. S. Gund and A. A. Jadhav, “Cloud computing security: Threats and countermeasures,” International Journal of Advanced Research in Science, Communication and Technology, vol. 3, no. 5, pp. 517–523, Jun. 2023.
R. Sihwail, K. Omar, and K. A. Zainol Ariffin, “A survey on malware analysis techniques: Static, dynamic, hybrid and memory analysis,” International Journal on Advanced Science, Engineering and Information Technology, vol. 8, no. 4–2, pp. 1662–1671, Sept. 2018.
P. Mishra, V. Varadharajan, U. Tupakula, and E. S. Pilli, “A detailed investigation and analysis of using machine learning techniques for intrusion detection,” IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 686–728, 2019.
T. G. Dietterich, “Ensemble methods in machine learning,” Multiple Classifier Systems, Jan. 2000, pp. 1–15.
M. Nawrocki, M. Wählisch, T. C. Schmidt, C. Keil, and J. Schönfelder, “A survey on honeypot software and data analysis,” arXiv, Aug. 22, 2016.
D. Gibert, C. Mateu, and J. Planes, “The rise of machine learning for detection and classification of malware: Research developments, trends and challenges,” Journal of Network and Computer Applications, vol. 153, Mar. 2020.
K. Shaukat, S. Luo, V. Varadharajan, I. A. Hameed, and M. Xu, “A survey on machine learning techniques for cyber security in the last decade,” IEEE Access, vol. 8, pp. 222310–222354, 2020.
L. Rokach, “Ensemble-based classifiers,” Artificial Intelligence Review, vol. 33, pp. 1–39, 2010.
B. Gregg, BPF Performance Tools. Boston, MA, USA: Addison-Wesley Professional, 2019.
L. Bohang et al., “Image steganalysis using active learning and hyperparameter optimization,” Scientific Reports, vol. 15, Mar. 2025.
Q. O. Ahmed, “Machine learning for intrusion detection in cloud environments: A comparative study,” Journal of Artificial Intelligence General Science, vol. 6, no. 1, pp. 550–563, Dec. 2024.
B. M. Khammas, “Ransomware detection using random forest technique,” ICT Express, vol. 6, no. 4, Dec. 2020.
Y. P and S. R, “Social engineering detection using behavioral data and form-usage patterns,” ITM Web of Conferences, vol. 85, 2026.
L. Breiman, “Random Forests,” Machine Learning, vol. 45, pp. 5–32, Oct. 2001.
V. N. Vapnik, “Introduction: Four periods in the research of the learning problem,” in The Nature of Statistical Learning Theory, 2nd ed. New York, NY, USA: Springer, 2000, pp. 1–15.
S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
M. Barabas, I. Homoliak, M. Drozd, and P. Hanacek, “Automated malware detection based on novel network behavioral signatures,” International Journal of Engineering and Technology, vol. 5, no. 2, pp. 249–253, Apr. 2013.
R. Shu, T. Xia, L. Williams, and T. Menzies, “Omni: Automated ensemble with unexpected models against adversarial evasion attack,” Empirical Software Engineering, vol. 27, 2022.
D. Mughole Kalimumbalo et al., “SecFedMDM-1: A federated learning-based malware detection model for interconnected cloud infrastructures,” IEEE Access, vol. 13, pp. 101246–101261, 2025.
S. Choudhury, M. Bojinov, J. P. Jue and G. Ishigaki, “Reinforcement learning-based E2E monitoring path selection for multi-domain optical networks,” GLOBECOM 2025 - 2025 IEEE Global Communications Conference, Taipei, Taiwan, 2025, pp. 3348–3353.
L. Ketari and M. A. Khanum, “A review of malicious code detection techniques for mobile devices,” International Journal of Computer Theory and Engineering, vol. 4, no. 2, pp. 212–216, Apr. 2012.
Z. Zhang, H. Al Hamadi, E. Damiani, C. Y. Yeun, and F. Taher, “Explainable artificial intelligence applications in cyber security: State-of-the-Art in research,” IEEE Access, vol. 10, pp. 93104–93139, 2022.
C. J. Fung and Q. Zhu, “FACID: A trust-based collaborative decision framework for intrusion detection networks,” Ad Hoc Networks, vol. 53, pp. 17–31, Dec. 2016.
C. Lo, T. Y. Win, Z. Rezaeifar, Z. Khan, and P. Legg, “LOTL-hunter: Detecting multi-stage living-off-the-land attacks in cyber-physical systems using decision fusion techniques with digital twins,” Future Generation Computer Systems, vol. 180, Jul. 2026.
C. Patil, S. A. Khan, M. Tahir K, and S. S. Khan, “Real time threat intelligence system for malware detection,” International Journal for Research in Applied Science and Engineering Technology, vol. 12, no. 4, pp. 5204–5208, Apr. 2024.
Y. Liu and Y. Wang, “A robust malware detection system using deep learning on API calls,” 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chengdu, China, 2019, pp. 1456–1460.
B. Pargi, S. Degadwala, and M. Joshi, “Hybrid malware analysis using static and dynamic techniques with machine learning,” International Journal of Scientific Research in Science, Engineering and Technology, vol. 13, no. 1, pp. 22–27, Jan. 2026.
M. Ogunbadejo, O. A. Ayilara-Adewale, and O. Alade, “Machine learning methods for intrusion detection: A comprehensive survey,” International Journal of Scientific Research and Management (IJSRM), vol. 13, no. 7, pp. 2446–2456, Jul. 2025.
M. Schofield et al., “Convolutional neural network for malware classification based on API call sequence,” Computer Science & Information Technology (CS & IT), pp. 85–98, Jan. 2021.
A.-M. Saadoon Balasim, S. T. Hasson, and M. S. Mohamood, “Deep feature mapping and ensemble learning for advanced IoT malware detection and classification,” Journal of Information Systems Engineering and Management, vol. 10, no. 21s, pp. 871–877, Mar. 2025.
