Abstract

The recording of knowledge through writing has enabled future generations to understand their cultural legacy and build a better future. However, ancient records were often created on plant leaves or other fragile materials. Although written documentation of human civilization dates back nearly 5,000 years, the quality of these handwritten manuscripts deteriorates over time due to ink fading and the biodegradation of materials such as paper. As a result, the preservation of historical documents remains a significant challenge. For several decades, computer scientists have been developing algorithms to recognize and digitize handwritten content. In this work, a residual neural network (ResNet284) is proposed for the recognition of handwritten English text and digits. Experiments were conducted on six benchmark datasets: Institute of Automated Machine Analysis Dataset (IAM) Handwritten dataset (115,320 words, segmented into approximately 576,600 characters), Kaggle English Handwritten (370,000 samples), Kaggle Digits (3,410 samples), Modified National Institute of Standards and Technology (MNIST) Digits (70,000 samples), Extended MNIST (EMNIST), treated as a unified dataset with subsets including Digits (350,000 samples comprising Digits-I with 280,000 and Digits-II with 70,000), EMNIST English Handwritten (ByClass: 814,255 characters; ByMerge: 814,255 characters; Balanced: 131,600 characters; and Letters: 145,600 characters), and the Centre of Excellence for Document Analysis and Recognition (CEDAR) dataset (100,000 samples). The proposed model achieved a test accuracy of 99.87% with an error rate of 0.13% on the combined digits and characters dataset comprising 2,768,780 samples. For the individual datasets, the model also demonstrated consistently high performance: 98.89% accuracy (1.11% error) on the IAM Handwritten dataset, 98.94% accuracy (1.06% error) on the Kaggle English Handwritten dataset, 98.91% accuracy (1.09% error) on the Kaggle Digits dataset, and 98.95% accuracy (1.05% error) on the MNIST Digits dataset. Similarly, the EMNIST English Handwritten dataset yielded 98.97% accuracy (1.03% error), while the CEDAR dataset resulted in 98.42% accuracy (1.58% error). Additionally, the combined digits dataset achieved 99.42% accuracy (0.58% error), and the combined characters dataset recorded 99.01% accuracy (0.99% error), further demonstrating the robustness and generalization capability of the proposed approach

Keywords

Handwritten text recognition, Residual neural network (ResNet284), Document digitization, Character classification, Digit recognition.

Cite this article

Sharma MK. Handwritten English text and digit recognition using ResNet284. International Journal of Advanced Technology and Engineering Exploration. 2025;12(132):1722-1748. DOI : 10.19101/IJATEE.2024.111101131

References

[1] Chandio AA, Asikuzzaman MD, Pickering MR, Leghari M. Cursive text recognition in natural scene images using deep convolutional recurrent neural network. IEEE Access. 2022; 10:10062-78.

[2] Jaderberg M, Vedaldi A, Zisserman A. Deep features for text spotting. In European conference on computer vision 2014 (pp. 512-28). Cham: Springer International Publishing.

[3] Jaderberg M, Simonyan K, Vedaldi A, Zisserman A. Reading text in the wild with convolutional neural networks. International Journal of Computer Vision. 2016; 116(1):1-20.

[4] Kaur R, Uppal M, Gupta D. A comprehensive and comparative study of handwriting recognition system. In renewable energy and sustainable e-mobility conference (RESEM) 2023 (pp. 1-6). IEEE.

[5] Agrawal V, Jagtap J, Kantipudi MP. Exploration of advancements in handwritten document recognition techniques. Intelligent Systems with Applications. 2024; 22:200358.

[6] Alkendi W, Gechter F, Heyberger L, Guyeux C. Advancements and challenges in handwritten text recognition: a comprehensive survey. Journal of Imaging. 2024; 10(1):1-30.

[7] Balci B, Saadati D, Shiferaw D. Handwritten text recognition using deep learning. CS23 1n: Convolutional Neural Networks for Visual Recognition, Stanford University, Course Project Report, Spring. 2017:752-9.

[8] Jia L, Shifa C, Qun G, Yiping Z, Lianrui Y. An empirical study of the influence of “pen-and-paper handwriting” on the recognition of English words by Chinese learners of English. Journal of Beijing International Studies University. 2023; 45(3):103-19.

[9] Almanea M. Deep learning in written Arabic linguistic studies: a comprehensive survey. IEEE Access. 2024:172196-233.

[10] Yeleussinov A, Amirgaliyev Y, Cherikbayeva L. Improving OCR accuracy for Kazakh handwriting recognition using GAN models. Applied Sciences. 2023; 13(9):1-13.

[11] Kumar N, Chikkmath A, BR GY, Naidu HR, Acharya D. Interpreting doctor notes using handwriting recognition and deep learning techniques: a survey. In international conference on advances in electronics, communication, computing and intelligent information systems (ICAECIS) 2023 (pp. 703-8). IEEE.

[12] De SNAF, Bezerra BL, De MGC, Toselli AH. Data augmentation for offline handwritten text recognition: a systematic literature review. SN Computer Science. 2024; 5(2):1-20.

[13] Ghanim M, Mohammed A, Sali A. Arabic/English handwritten digits recognition using MLPS, CNN, RF, and CNN-RF. Al-Rafidain Engineering Journal. 2023; 28(2):252-60.

[14] Pippi V, Cascianelli S, Kermorvant C, Cucchiara R. How to choose pretrained handwriting recognition models for single writer fine-tuning. In international conference on document analysis and recognition 2023 (pp. 330-47). Cham: Springer Nature Switzerland.

[15] K JM, H PC, D VA, Mali S. A comprehensive literature review on air-written online handwritten recognition. International Journal of Computing and Digital Systems. 2024; 15(1):307-22.

[16] Wang Y, Yang J, Cai M, Liu X, Lu K, Lou Y, et al. Application of optimized convolutional neural networks for early aided diagnosis of essential tremor: automatic handwriting recognition and feature analysis. Medical Engineering & Physics. 2023; 113:103962.

[17] Medjram S, Eglin V, Bres S. Study’s system: a three-stage system for children handwriting recognition and spelling error detection. Multimedia Tools and Applications. 2024; 83(36):83789-803.

[18] Pan Y, Fan D, Wu H, Teng D. A new dataset for Mongolian online handwritten recognition. Scientific Reports. 2023; 13(1):1-16.

[19] Anoop R, Kiran N, Raj BP, Kodipalli A, Rao T, Rohini BR. MNSIT handwritten digit recognition using machine learning classification algorithms. In world conference on communication & computing (WCONF) 2023 (pp. 1-5). IEEE.

[20] Masruroh SU, Syahid MF, Munthaha F, Muharram AT, Putri RA. Deep convolutional neural networks transfer learning comparison on Arabic handwriting recognition system. JOIV: International Journal on Informatics Visualization. 2023; 7(2):330-7.

[21] Shi B, Bai X, Yao C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016; 39(11):2298-304.

[22] Albahli S, Nawaz M, Javed A, Irtaza A. An improved faster-RCNN model for handwritten character recognition. Arabian Journal for Science and Engineering. 2021; 46(9):8509-23.

[23] Ali S, Shaukat Z, Azeem M, Sakhawat Z, Mahmood T, Ur RK. An efficient and improved scheme for handwritten digit recognition based on convolutional neural network. SN Applied Sciences. 2019; 1(9):1-9.

[24] Aly S, Almotairi S. Deep convolutional self-organizing map network for robust handwritten digit recognition. IEEE Access. 2020; 8:107035-45.

[25] Nguyen KC, Nguyen CT, Nakagawa M. A semantic segmentation-based method for handwritten Japanese text recognition. In 17th international conference on frontiers in handwriting recognition (ICFHR) 2020 (pp. 127-32). IEEE.

[26] Alif MA, Ahmed S, Hasan MA. Isolated Bangla handwritten character recognition with convolutional neural network. In 20th international conference of computer and information technology (ICCIT) 2017 (pp. 1-6). IEEE.

[27] Wu X, Chen Q, You J, Xiao Y. Unconstrained offline handwritten word recognition by position embedding integrated resnets model. IEEE Signal Processing Letters. 2019; 26(4):597-601.

[28] Zahra SB, Moaen S, Munir S, Hassan A, Nadeem A, Farooq MS. Optical handwritten with character recognition. In international conference on innovative computing (ICIC) 2021 (pp. 1-8). IEEE.

[29] Al-rawi M, Valveny E, Karatzas D. Can one deep learning model learn script-independent multilingual word-spotting? In international conference on document analysis and recognition (ICDAR) 2019 (pp. 260-7). IEEE.

[30] Bouibed ML, Nemmour H, Chibani Y, Hafiane A. Hybrid combination of convolutional neural network features for robust writer retrieval of historical documents. In IEEE/ACS 17th international conference on computer systems and applications (AICCSA) 2020 (pp. 1-5). IEEE.

[31] Purohit A, Chauhan SS. A literature survey on handwritten character recognition. International Journal of Computer Science and Information Technologies (IJCSIT). 2016; 7(1):1-5.

[32] Das A, Mohanty MN. Handwritten Odia numeral recognition using combined CNN-RNN. International Journal of Grid and Utility Computing. 2023; 14(4):382-8.

[33] Wang Z, Muhammat M, Yadikar N, Aysa A, Ubul K. Advances in offline handwritten signature recognition research: a review. IEEE Access. 2023; 11:120222-36.

[34] Parameshachari BD, Ashok A, Reddy H. Comparative analysis of handwritten text recognition using CNN and SVM. In international conference on distributed computing and electrical circuits and electronics (ICDCECE) 2023 (pp. 1-5). IEEE.

[35] Bhatti A, Arif A, Khalid W, Khan B, Ali A, Khalid S, et al. Recognition and classification of handwritten Urdu numerals using deep learning techniques. Applied Sciences. 2023; 13(3):1-15.

[36] Inunganbi S. A systematic review on handwritten document analysis and recognition. Multimedia Tools and Applications. 2024; 83(2):5387-413.

[37] El MM. An effective combination of convolutional neural network and support vector machine classifier for Arabic handwritten recognition. Automatic Control and Computer Sciences. 2023; 57(3):267-75.

[38] Davoudi H, Traviglia A. Discrete representation learning for handwritten text recognition. Neural Computing and Applications. 2023; 35(21):15759-73.

[39] Dhiaf M, Rouhou AC, Kessentini Y, Salem SB. MSdocTr-lite: a lite transformer for full page multi-script handwriting recognition. Pattern Recognition Letters. 2023; 169:28-34.

[40] Momeni S, Babaali B. A transformer-based approach for Arabic offline handwritten text recognition. Signal, Image and Video Processing. 2024; 18(4):3053-62.

[41] Zhang A, Lipton ZC, Li M, Smola AJ. Dive into deep learning. Cambridge University Press; 2023.

[42] Diederik K. Adam: a method for stochastic optimization. In international conference on learning representations (ICLR) 2015 (pp.1-5).

[43] Mao A, Mohri M, Zhong Y. Cross-entropy loss functions: theoretical analysis and applications. In international conference on machine learning 2023 (pp. 1-26). PMLR.

[44] Winiarti S, Ismi DP, Prahara A. Image enhancement using piecewise linear contrast stretch methods based on unsharp masking algorithms for leather image processing. In 3rd international conference on science in information technology (ICSITech) 2017 (pp. 669-73). IEEE.

[45] Gedraite ES, Hadad M. Investigation on the effect of a gaussian blur in image filtering and segmentation. In proceedings of international symposium on electronics in marine 2011 (pp. 393-6). IEEE.

[46] Marti UV, Bunke H. A full English sentence database for off-line handwriting recognition. In proceedings of the fifth international conference on document analysis and recognition 1999 (pp. 705-8). IEEE.

[47] Buscema M. Metanet: the theory of independent judges. Substance Use & Misuse. 1998; 33(2):439-61.

[48] Llorens D, Prat F, Marzal A, Vilar JM, Castro MJ, Amengual JC, et al. The ujipenchars database: a pen-based database of isolated handwritten characters. International conference on language resources and evaluation 2008 (pp. 2647-51).

[49] Williams BH, Toussaint M, Storkey AJ. Extracting motion primitives from natural handwriting data. In international conference on artificial neural networks 2006 (pp. 634-43). Berlin, Heidelberg: Springer Berlin Heidelberg.

[50] Botta M, Giordana A, Saitta L. Learning fuzzy concept definitions. In proceedings second international conference on fuzzy systems 1993 (pp. 18-22). IEEE.

[51] https://www.kaggle.com/. Accessed 20 October 2025.

[52] Li W, He X, Tang C, Wu K, Chen X, Yu S, et al. Handwritten numbers and English characters recognition system. In the Euro-China conference on intelligent data analysis and applications 2016 (pp. 145-54). Cham: Springer International Publishing.

[53] Roy P, Ghosh S, Pal U. A CNN based framework for unistroke numeral recognition in air-writing. In 16th international conference on frontiers in handwriting recognition (ICFHR) 2018 (pp. 404-9). IEEE.

[54] Tabassum S, Abedin N, Rahman MM, Rahman MM, Ahmed MT, Islam R, et al. An online cursive handwritten medical words recognition system for busy doctors in developing countries for ensuring efficient healthcare service delivery. Scientific Reports. 2022; 12(1):1-13.

[55] Khan FA, Khelifi F, Tahir MA, Bouridane A. Dissimilarity gaussian mixture models for efficient offline handwritten text-independent identification using SIFT and root SIFT descriptors. IEEE Transactions on Information Forensics and Security. 2018; 14(2):289-303.

[56] Chakraborty A, De R, Malakar S, Schwenker F, Sarkar R. Handwritten digit string recognition using deep autoencoder based segmentation and resnet based recognition approach. In 25th international conference on pattern recognition (ICPR) 2021 (pp. 7737-42). IEEE.

[57] Saidulu N, Monsley KA, Yadav KS, Laskar RH. Exploration of deep convolutional neural networks (via transfer learning) for handwritten character recognition. In second international conference on power, control and computing technologies (ICPC2T) 2022 (pp. 1-6). IEEE.

[58] Singh S, Paul A, Arun M. Parallelization of digit recognition system using deep convolutional neural network on CUDA. In third international conference on sensing, signal processing and security (ICSSS) 2017 (pp. 379-83). IEEE.

[59] Jayasundara V, Jayasekara S, Jayasekara H, Rajasegaran J, Seneviratne S, Rodrigo R. Textcaps: handwritten character recognition with very small datasets. In winter conference on applications of computer vision (WACV) 2019 (pp. 254-62). IEEE.

[60] Shuvo MI, Akhand MA, Siddique N. Handwritten numeral recognition through superimposition onto printed form. Journal of King Saud University-Computer and Information Sciences. 2022; 34(9):7751-64.