A Data-Driven Automata-Theoretic Framework for Emotional State Recognition Through Music Interaction Dynamics
DOI:
https://doi.org/10.46610/JoSCCI.2026.v03i01.003Keywords:
Affective computing, Automata theory, Data-driven models, Emotional state recognition, Finite state automata, Music interaction dynamicsAbstract
Music is an effective channel of expression of emotions, and the patterns of interaction between users and the music platforms have provided a richer source of behavioural indicators of emotional state. The present paper presents a data-driven Automata-Theoretic Framework of emotional state recognition via music interaction dynamics, a machine learning-based approach to formal automata theory to obtain structurally, interpretably, and adaptively modelled emotions. The framework relies on the data of user-music interaction, including the duration of listening, skip patterns, number of replays, preferred tempo, and genre changeover to derive behavioural characteristics which are of significance. These features are learned with data-driven learning models to give probabilistic emotion indicators that are then mapped to discrete emotional states with the help of a finite automaton. The automata-theoretic layer formalises emotion changes, which guarantees temporal consistency and logical correctness in emotion changes, and overcomes the weakness of the purely statistical approach that is frequently difficult to interpret. The given framework defines emotions as states, and interaction-driven cues as transition symbols; therefore, by doing so, it will be able to capture the levels of short-term affective reactions, as well as long-term emotional tendencies. Through experimental analysis, the hybrid approach proves to be more robust and less noisy (predictive) and more explainable than standalone machine learning models. The suggested framework can be used in music recommendation systems, affect-sensitive human-computer interaction, and mental health monitoring and adaptive entertainment systems. In general, this paper demonstrates the opportunities of the data-based intelligence and automata theory integration to provide a stable and interpretable emotional state recognition.
References
M. Wang and T. Jiang, “A social network analysis of user-organised community on digital music platform,” 2024 IEEE/ACIS 27th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Beijing, China, 2024, pp. 62–65.
A. K. Alimuradov, A. Y. Tychkov, D. S. Dudnikov and P. P. Churakov, “Software and hardware implementation of a system to detect and classify human emotional states from speech signals under extreme conditions,” 2023 5th International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA), Lipetsk, Russian Federation, 2023, pp. 241–246.
S. Nam, C. Park, J. L. Bautista and H. Shin, “A study on physiological data collection and validation in emotion recognition,” 2024 IEEE Consumer Life Tech (ICLT), Sydney, Australia, 2024, pp. 1–4.
K. Klüber and E. Roesler, “The influence of embodiment on the emotional contagion of affective speech in HRI,” 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Melbourne, Australia, 2025, pp. 1408–1412.
A. Saluja, A. Panda, S. Kesavan and N. Panda, “Design and implementation of a movie rental website using automata theory,” 2024 5th IEEE Global Conference for Advancement in Technology (GCAT), Bangalore, India, 2024, pp. 1–7.
Y. Xiang, “Application of computer computer music software platformin music teaching in colleges and universities,” 2022 3rd International Conference on Education, Knowledge and Information Management (ICEKIM), Harbin, China, 2022, pp. 291–294.
J. He, “Analysis of the online product demand trend based on the comment recognition model,” 2021 IEEE 5th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Xi'an, China, 2021, pp. 1181–1186.
D. Iren, E. Yildirim and K. Shingjergji, “Ethical risks, concerns, and practices of affective computing: A thematic analysis,” 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Cambridge, MA, USA, 2023, pp. 1–4.
U. Kamble, S. Nalawade, T. Mahadik, Y. Salunke, S. Dedgaonkar and P. Shelke, “Survey of application of automata theory in natural language processing,” 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS), Manama, Bahrain, 2024, pp. 1–4.
D. Feng, T. Wang, S. Zhou, Y. Li and G. -Q. Su, “A data-driven modeling method for power quality disturbance sources in distribution networks,” 2021 International Conference on Power System Technology (POWERCON), Haikou, China, 2021, pp. 2393–2398.
K. Kishi, K. Sato and T. Kosaka, “Speech emotion recognition based on linguistic features of spoken language,” 2024 IEEE 13th Global Conference on Consumer Electronics (GCCE), Kitakyushu, Japan, 2024, pp. 1157–1160.
K. Zhang, J. Tian, J. Han and Q. Yuan, “A lightweight model-driven MES simulation framework based on probabilistic finite automata,” 2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP), Xi'an, China, 2022, pp. 1093–1096.
J. Deng and F. Ren, “Multi-label emotion detection via emotion-specified feature extraction and emotion correlation learning,” in IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 475–486, 1 Jan.-Mar. 2023.
E. Ganiti-Roumeliotou, I. Ziogas, S. B. Dias, G. Alhussein, H. F. Jelinek and L. J. Hadjileontiadis, “Beyond the game: Multimodal emotion recognition before, during, and after gameplay,” 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2024, pp. 1–7.
E. B. Varghese, S. M. Thampi and S. Berretti, “A psychologically inspired fuzzy cognitive deep learning framework to predict crowd behavior,” in IEEE Transactions on Affective Computing, vol. 13, no. 2, pp. 1005–1022, 1 Apr.-Jun. 2022.
S. K. W. Hwooi, A. Othmani and A. Q. M. Sabri, “Deep learning-based approach for continuous affect prediction from facial expression images in valence-arousal space,” in IEEE Access, vol. 10, pp. 96053–96065, 2022.
I. Predoaia, “Towards systematic engineering of hybrid graphical-textual domain-specific languages,” 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Västerås, Sweden, 2023, pp. 153–158.
S. Xu, Y. Cao, H. Wang, W. Wang, D. Yang and X. Cai, “Oscillation mode identification of self-synchronized voltage source doubly-fed wind turbine based on continuous wavelet transform,” 2023 IEEE 14th International Symposium on Power Electronics for Distributed Generation Systems (PEDG), Shanghai, China, 2023, pp. 180–184.
L. B. Laskov and V. M. Georgieva, “Analysis of amplitude modulation and demodulation in MATLAB Simulink environment,” 2021 56th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), Sozopol, Bulgaria, 2021, pp. 223–226.
S. Braun and H. Gamper, “Multi-label audio classification with a noisy zero-shot teacher,” 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC), Aalborg, Denmark, 2024, pp. 240–244.
Priyadarshini, S. Greiner, M. Massierer and O. -E. -K. Aktouf, “Software architecture based inter-feature analysis of safety and security interactions,” 2024 19th European Dependable Computing Conference (EDCC), Leuven, Belgium, 2024, pp. 165–168.
S. Xie et al., “Wireless NIR four-quadrant detector system based on PbS Collodial quantum dots,” in IEEE Electron Device Letters, vol. 46, no. 2, pp. 215–218, Feb. 2025.
S. S. Raju, B. Wang, K. Mehta, M. Xiao, Y. Zhang and H. -Y. Wong, “Application of noise to avoid overfitting in TCAD augmented machine learning,” 2020 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), Kobe, Japan, 2020, pp. 351–354.
R. Rikame, M. Kr. Ranjan, M. Jadhav, S. Wankhede, S. Bankar, and P. Bachhav, “A hybrid automata-driven machine learning framework for real-time energy optimization in smart buildings,” First International Conference on Engineering and Technology for a Sustainable Future, EPJ Web of Conferences, vol. 328, Jun. 2025.
H. Li, Y. Li, W. Li, and Y. Liu, “Stability analysis of block logical dynamical systems and its application in logical networks with time delay,” IEEE Transactions on Automatic Control, vol. 69, no. 10, pp. 7211–7215, Oct. 2024.
A. Patil, M. Ranjan, K. Deore, P. Sonje, K. Patil, and R. Patil, “Hybrid automata-based control framework for real-time optimization in space-based solar power transmission,” First International Conference on Engineering and Technology for a Sustainable Future, EPJ Web of Conferences, vol. 328, Jun. 2025.
W. Lei, L. Liu and L. Liu, “Spatio-temporal structure consistency for semi-supervised medical image classification,” ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023, pp. 1–5.
A. K. Alimuradov et al., “Research into the influence of informative speech parameters on the reliability of RNN-based classification of human emotional states,” 2024 26th International Conference on Digital Signal Processing and its Applications (DSPA), Moscow, Russian Federation, 2024, pp. 1–6.
X. Yang, M. J. Zuo and Z. Tian, “Normalization of gearbox vibration signal for tracking tooth crack severity progression under time-varying operating conditions,” 2020 Asia-Pacific International Symposium on Advanced Reliability and Maintenance Modeling (APARM), Vancouver, BC, Canada, 2020, pp. 1–6.
Palak, S. Chadha, P. Sharma and V. S. Babu Chilluri, “Emotion recognition using a hybrid model of transformer and CNN-LSTM for improved performance,” 2025 3rd International Conference on Disruptive Technologies (ICDT), Greater Noida, India, 2025, pp. 866–870.
