skip to main content

A Hybrid VGG19-XGBoost Framework with SSIM-Based Feedback for Low-Resource Handwritten Digit Recognition in Digital Learning Systems

1Department of Fashion Communication, National Institute of Fashion Technology, India

2Engineer 1, Numerator, India

3Software Development Officer, Digital Health and Care Wales, United Kingdom

Received: 25 Sep 2025; Revised: 1 Apr 2026; Accepted: 18 May 2026; Published: 25 May 2026.
Open Access Copyright (c) 2026 The authors. Published by Department of Informatics Universitas, Diponegoro
Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Citation Format:
Abstract

Handwritten digit recognition is an important component of intelligent educational interfaces, particularly in early learning, digital assessment, and handwriting feedback systems. However, many deep learning-based recognition models rely on large benchmark datasets, whereas practical educational environments may involve limited and heterogeneous learner-generated samples. This study presents a technical feasibility analysis of a hybrid handwritten-digit recognition framework that combines VGG19-based deep feature extraction, XGBoost classification, and Structural Similarity Index Measure (SSIM)-based visual-similarity feedback. A small handwritten-digit dataset was used to simulate a constrained-data setting, with augmentation applied only to the training data to reduce the risk of overfitting. The proposed VGG19-XGBoost pipeline was evaluated against baseline CNN-based models using accuracy, precision, recall, and F1-score, with SSIM used as a supplementary metric to assess the structural similarity between learner input and reference digit forms. The experimental results indicate that the hybrid approach provides stable preliminary classification performance under limited-data conditions and that SSIM can support interpretable visual feedback for handwriting evaluation. However, due to the small number of original samples, the findings should be interpreted as evidence of feasibility rather than as generalizable performance claims. Future work should involve larger real-world datasets, teacher-validated scoring rubrics, and deployment-oriented evaluation in digital learning environments.

Keywords: Handwritten Digit Recognition; Transfer Learning; VGG19; XG Boost; Structural Similarity Index; Educational Technology Feasibility

Article Metrics:

  1. A. Dehghanian and V. Ghods, “Farsi Handwriting Digit Recognition Based on Convolutional Neural Networks,” in 2018 6th International Symposium on Computational and Business Intelligence (ISCBI), Aug. 2018, pp. 65–68. doi: 10.1109/ISCBI.2018.00022
  2. C. Zhang, Z. Zhou, and L. Lin, “Handwritten Digit Recognition Based on Convolutional Neural Network,” in 2020 Chinese Automation Congress (CAC), Nov. 2020, pp. 7384–7388. doi: 10.1109/CAC51589.2020.9326781
  3. S. Ahlawat and R. Rishi, “Handwritten Digit Recognition using Adaptive Neuro-Fuzzy System and Ranked Features,” in 2018 International Conference on Computing, Power and Communication Technologies (GUCON), Sep. 2018, pp. 1128–1132. doi: 10.1109/GUCON.2018.8675013
  4. Z. Kayumov and D. Tumakov, “Convolution Neural Network Learning Features for Handwritten Digit Recognition,” in 2020 IEEE East-West Design & Test Symposium (EWDTS), Sep. 2020, pp. 1–5. doi: 10.1109/EWDTS50664.2020.9224822
  5. M. Rajalakshmi, P. Saranya, and P. Shanmugavadivu, “Pattern Recognition-Recognition of Handwritten Document Using Convolutional Neural Networks,” in 2019 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), Apr. 2019, pp. 1–7. doi: 10.1109/INCOS45849.2019.8951342
  6. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” ArXiv, Apr. 10, 2015. https://arxiv.org/abs/1409.1556
  7. Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004, doi: 10.1109/TIP.2003.819861
  8. R. Sethi and I. Kaushik, “Hand Written Digit Recognition using Machine Learning,” in 2020 IEEE 9th International Conference on Communication Systems and Network Technologies (CSNT), Apr. 2020, pp. 49–54. doi: 10.1109/CSNT48778.2020.9115746
  9. A. Choudhury, H. S. Rana, and T. Bhowmik, “Handwritten Bengali Numeral Recognition using HOG Based Feature Extraction Algorithm,” in 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), Feb. 2018, pp. 687–690. doi: 10.1109/SPIN.2018.8474215
  10. O. Nyide and M. V. Gwetu, “Handwritten Digit Classification using Iterative Haar-like Operators and Neural Networks,” in 2019 International Conference on Advances in Big Data, Computing and Data Communication Systems (icABCD), Aug. 2019, pp. 1–5. doi: 10.1109/ICABCD.2019.8850998
  11. K. T. Islam, G. Mujtaba, R. G. Raj, and H. F. Nweke, “Handwritten digits recognition with artificial neural network,” in 2017 International Conference on Engineering Technology and Technopreneurship (ICE2T), Sep. 2017, pp. 1–4. doi: 10.1109/ICE2T.2017.8215993
  12. M. Shopon, N. Mohammed, and M. A. Abedin, “Image augmentation by blocky artifact in Deep Convolutional Neural Network for handwritten digit recognition,” in 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR), 2017, pp. 1–6. doi: 10.1109/ICIVPR.2017.7890867
  13. K. V. P. Nandan, M. Panda, and S. Veni, “Handwritten Digit Recognition Using Ensemble Learning,” in 2020 5th International Conference on Communication and Electronics Systems (ICCES), Jun. 2020, pp. 1008–1013. doi: 10.1109/ICCES48766.2020.9137933
  14. A. Shrivastava, I. Jaggi, S. Gupta, and D. Gupta, “Handwritten Digit Recognition Using Machine Learning: A Review,” in 2019 2nd International Conference on Power Energy, Environment and Intelligent Control (PEEIC), Oct. 2019, pp. 322–326. doi: 10.1109/PEEIC47157.2019.8976601
  15. B. Melis Ozyildirim and M. Avci, “Handwritten Digits Classification with Generalized Classifier Neural Network,” in 2018 Innovations in Intelligent Systems and Applications Conference (ASYU), Oct. 2018, pp. 1–3. doi: 10.1109/ASYU.2018.8553999
  16. M. Boukrouh, M. Lashab, A. Goutas, and S. Ouchtati, “Recognition of Handwritten Digits Using Neuro-Markovian Technique,” in 2018 International Conference on Signal, Image, Vision and their Applications (SIVA), Nov. 2018, pp. 1–5. doi: 10.1109/SIVA.2018.8661148
  17. W. He, T. Zhou, Y. Xiang, Y. Lin, J. Hu, and R. Bao, “Deep Learning in Image Classification: Evaluating VGG19’s Performance on Complex Visual Data,” in 2025 5th International Conference on Neural Networks, Information and Communication Engineering (NNICE), Jan. 2025, pp. 261–265. doi: 10.1109/NNICE64954.2025.11064681
  18. M. Rahman, Y. Cao, X. Sun, B. Li, and Y. Hao, “Deep pre-trained networks as a feature extractor with XGBoost to detect tuberculosis from chest X-ray,” Comput. Electr. Eng., vol. 93, p. 107252, Jul. 2021, doi: 10.1016/j.compeleceng.2021.107252

Last update:

No citation recorded.

Last update: 2026-05-29 09:36:48

No citation recorded.