Automated Handwritten Text Analysis and Recognition Using Neural Networks
Keywords:
Bidirectional Long Short-Term Memory (BiLSTM), Connectionist Temporal Classification (CTC), Convolutional Neural Network (CNN), Deep learning, Handwritten text recognition, Neural network, Optical character recognitionAbstract
Handwritten text recognition (HTR) is a difficult problem in Optical Character Recognition (OCR) systems, owing to the extreme variability of handwriting styles, degradation of image quality, and diverse character appearances. The present paper proposes a comprehensive framework based on deep learning techniques, which incorporates Convolutional Neural Networks (CNNs), Bidirectional Long Short-Term Memory (BiLSTM) models, and Connectionist Temporal Classification (CTC) for effective handwritten text recognition. The hybrid model utilizes CNNs to learn spatial features from handwritten text images, BiLSTMs to learn sequential relationships between characters, and CTC to learn sequences directly from images without requiring explicit character segmentation. The model incorporates various preprocessing techniques to improve its performance, including image normalization, noise reduction, and data augmentation, to achieve robustness against diverse handwriting styles. The performance of the model has been evaluated on benchmark datasets, achieving state-of-the-art performance with a character error rate of 2.95% and a word error rate of less than 7%. The model achieves an accuracy of 94% during validation, with efficient computation to suit practical scenarios. The present study contributes to the advancement of automated document digitization, accessibility for the visually impaired, efficient data processing in banking, healthcare, and education, among other areas.
References
A. A.-Fahandari, E. Shabaninia, F. A.-Zeydabadi, and H. N.-Pour, “A comprehensive survey of transformers in text recognition: Techniques, challenges, and future directions,” ACM Computing Surveys, vol. 58, no. 5, pp. 1–42, 2025.
A. Ansari, B. Kaur, M. Rakhra, A. Singh, and D. Singh, “Handwritten text recognition using deep learning algorithms,” 2022 4th International Conference on Artificial Intelligence and Speech Technology (AIST), Delhi, India, 2022, pp. 1–6.
M. Ayadi, N. Masmoudi, L. Almuqren, H. S. Alshahrani, and R. O. Aljohani, “Designing a novel CNN–LSTM-based model for Arabic handwritten character recognition for the visually impaired person,” Journal of Disability Research, vol. 4, no. 1, pp. 20240080, 2025.
G. Bastas, K. Kritsis, and V. Katsouros, “Air-writing recognition using deep convolutional and recurrent neural network architectures,” 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), Dortmund, Germany, 2020, pp. 7–12.
P. M. Bhagyashree, L. K. Likhitha, and D. S. Rajesh, “Handwritten digit recognition using deep learning,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 7, no. 4, pp. 153–158, Jul.–Aug. 2021.
D. Coquenet, C. Chatelain, and T. Paquet, “End-to-end handwritten paragraph text recognition using a vertical attention network,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 508–524, Jan. 2023.
R. P. Das, R. Dey, R. Mondal, R. Gangopadhyay, R. Dutta and J. Piri, "A Deep Learning-Driven OCR Framework for Medical Prescription Analysis Using Hybrid Preprocessing and Multi-Model Architecture," 2025 2nd International Conference on Circuits, Power and Intelligent Systems (CCPIS), Bhubaneswar, India, 2025, pp. 1-6.
S. L. Dissanayake and M. Fernando, "Attention-driven handwritten identification: A segmenation-free framework with diffusion based data augmentation," 2025 International Conference on Advanced Machine Learning and Data Science (AMLDS), Tokyo, Japan, 2025, pp. 242-247.
M. T. Equbal and S. Site, “Few-shot Hindi handwritten text recognition using meta-optimized ResNet-Transformer networks,” International Journal of Scientific Research in Engineering and Management, vol. 9, no. 8, pp. 1–17, 2025.
A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, "A Novel Connectionist System for Unconstrained Handwriting Recognition," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855-868, May 2009.
H. H. Hassan and A. Gülcü, "Handwritten Text Recognition using Deep Learning Methods," 2023 7th International Electromagnetic Compatibility Conference (EMC Turkiye), İstanbul, Turkiye, 2023, pp. 1-7.
I. Hussain, R. Ahmad, K. Ullah, S. Muhammad, R. Elhassan, and I. Syed, “Deep learning-based recognition system for Pashto handwritten text: Benchmark on PHTI,” PeerJ Computer Science, vol. 10, no. e1925, pp. 1-13, 2024.
J. Jebadurai, I. J. Jebadurai, G. J. L. Paulraj, and S. V. Vangeepuram, "Handwritten Text Recognition and Conversion Using Convolutional Neural Network (CNN) Based Deep Learning Model," 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2021, pp. 1037-1042.
S. Katoch, M. Rakhra, and D. Singh, "Recognition of Handwritten English Character Using Convolutional Neural Network," 2022 4th International Conference on Artificial Intelligence and Speech Technology (AIST), Delhi, India, 2022, pp. 1-6.
N. Kumar, A. Chikkmath, G. Y. B R, H. R. Naidu, and D. Acharya, "Interpreting Doctor notes using handwriting recognition and deep learning techniques: A survey," 2023 International Conference on Advances in Electronics, Communication, Computing and Intelligent Information Systems (ICAECIS), Bangalore, India, 2023, pp. 703-708.
Y. Li et al., "Fast and Robust Online Handwritten Chinese Character Recognition With Deep Spatial and Contextual Information Fusion Network," in IEEE Transactions on Multimedia, vol. 25, pp. 2140-2152.
X. Liu, B. Hu, Q. Chen, X. Wu, and J. You, "Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4637-4648, Nov. 2020.
N. -T. Ly, C. -T. Nguyen, K. -C. Nguyen, and M. Nakagawa, "Deep Convolutional Recurrent Network for Segmentation-Free Offline Handwritten Japanese Text Recognition," 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan, 2017, pp. 5-9.
M. G. Mahdi, A. Sleem, I. M. Elhenawy, and S. Safwat, “Enhancing the recognition of handwritten Arabic characters through hybrid convolutional and bidirectional recurrent neural network models,” Sustainable Machine Intelligence Journal, vol. 9, no. 1, pp. 34-56, 2024.
J. Michael, R. Labahn, T. Grüning, and J. Zöllner, "Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition," 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, NSW, Australia, 2019, pp. 1286-1293.
A. F. de Sousa Neto, B. L. D. Bezerra, A. H. Toselli, and E. B. Lima, "HTR-Flor: A Deep Learning System for Offline Handwritten Text Recognition," 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Porto de Galinhas, Brazil, 2020, pp. 54-61.
R. Malhotra and M. T. Addis, "Handwritten Amharic Word Recognition With Additive Attention Mechanism," in IEEE Access, vol. 12, pp. 114645-114657, 2024.
K. Murugesh, K. Sudharson, S. T. Kumar, R. Sanjiv, K. R. M. Raj, and R. Santhiya, "Swintrocr: A Transformer-Based Approach for High-Accuracy Tamil Text Recognition," 2025 3rd International Conference on Artificial Intelligence and Machine Learning Applications Theme: Healthcare and Internet of Things (AIMLA), Namakkal, India, 2025, pp. 1-6.
R. Dodda, S. B. Reddy, A. C. Naik, and V. Gaddam, “A study on handwritten text recognition classification using diverse deep learning techniques and computation of CTC loss,” CVR Journal of Science and Technology, vol. 26, no. 1, pp. 107–111, Jun. 2024.
K. C. Nguyen, C. T. Nguyen, and M. Nakagawa, "A Semantic Segmentation-based Method for Handwritten Japanese Text Recognition," 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), Dortmund, Germany, 2020, pp. 127-132.
