DOI: https://doi.org/10.14710/vol16iss1pp%25p

Comparative Evaluation of E-Learning Satisfaction: Divergence between SUS, PIECES, and EUCS Frameworks

*Rahmad Hidayat  -  Information System, ITB STIKOM Bali, Jl. Raya Puputan No.86, Dangin Puri Klod, Kec. Denpasar Tim., Kota Denpasar, Bali 80234, Indonesia
Open Access Copyright (c) 2026 Jurnal Sistem Informasi Bisnis

Citation Format:
Abstract
The deployment of e-learning systems in educational institutions is more concerned with speed than with the system's usability. In this study, we focused on the satisfaction and usability of Jimbaran Bali Global SMK elearning system. We applied the System Usability Scale (SUS) and End User Computing Satisfaction (EUCS) models alongside the PIECES framework. These methods have generally been applied separately, resulting in a limited understanding of the relationship between usability, system performance, and user satisfaction. Furthermore, studies that integrate and compare SUS, PIECES, and EUCS within a single evaluation framework, particularly in the Indonesian educational context, remain limited. The comparison of these methods is necessary because SUS focuses on usability, PIECES evaluates system and operational performance, and EUCS measures end-user satisfaction. Therefore, this study aims to evaluate the e-learning system of SMK TI Bali Global Jimbaran by comparing and integrating these three methods to provide a more comprehensive assessment of e-learning effectiveness. Having conducted both validation and reliability on the instrument, we analyzed data from 116 respondents, The results showed a discrepancy in usability and satisfaction, Satisfaction with PIECES and EUCS was positive, whereas the SUS score of 57.39 (Marginal–Low) is indicative of a significant problem in system usability. The findings show that users appreciated the system, but lacked the ability to use the system. These findings show that the system responsiveness, interface, and service quality must be improved to increase the positive perceptions of users and achieve sustainable e-learning.

Note: This article has supplementary file(s).

Fulltext |  Data Set
Dataset
Subject Dataset, SUS, PIECES, EUCS
Type Data Set
  Download (35KB)    Indexing metadata
 Data Analysis
Calculate SUS
Subject SUS
Type Data Analysis
  Download (41KB)    Indexing metadata
 Research Instrument
Calculate PIECES
Subject PIECES
Type Research Instrument
  Download (78KB)    Indexing metadata
 Research Instrument
Calculate EUCS
Subject EUCS
Type Research Instrument
  Download (64KB)    Indexing metadata
 Data Analysis
Output Validity and Reliability SUS
Subject Validity and Reliability SUS
Type Data Analysis
  Download (11KB)    Indexing metadata
 Data Analysis
Output Validity and Reliability PIECES
Subject Validity and Reliability SUS
Type Data Analysis
  Download (52KB)    Indexing metadata
 Data Analysis
Output Validity and Reliability EUCS
Subject Validity and Reliability SUS
Type Data Analysis
  Download (42KB)    Indexing metadata
Email colleagues
Keywords: E-learning Evaluation; Usability Satisfaction Gap; SUS; PIECES Framework; EUCS

Article Metrics:

Article Info
Section: Research Articles
Language : EN
Most cited articles
Sistem Informasi Pengukuran Kinerja Pada Perkebunan Kelapa Sawit Dengan Menggunakan Metode Balanced Scorecard Sistem Informasi Perpustakaan Berbasis Web Application Penerapan Metode AHP dan Fuzzy Topsis Untuk Sistem Pendukung Keputusan Promosi Jabatan Perencanaan Infrastruktur Teknologi Informasi pada Bank dengan Framework TOGAF Studi Pengamanan Login Pada Sistem Informasi Akademik Menggunakan Otentifikasi One Time Password Berbasisis SMS dengan Hash MD5 More cited articles
  1. ’Aisy, R., Mursityo, Y. T., & Wijoyo, S. H. (2024). Evaluasi Usability Aplikasi Mobile Sampingan Menggunakan Metode Usability Testing dan System Usability Scale (SUS). Jurnal Teknologi Informasi Dan Ilmu Komputer, 11(1), 19–26. https://doi.org/10.25126/jtiik.20241116613
  2. Al Waaidhoh, N., Sediyono, E., & Hartomo, K. D. (2020). Analisis Faktor E-Learning Readiness dengan Menggunakan Principal Component Analysis. JURNAL SISTEM INFORMASI BISNIS, 10(1), 73–83. https://doi.org/10.21456/vol10iss1pp73-83
  3. Al-Rikabi, Y. K., & Ali Montazer, G. (2023). Extracting the main aspects of e-Learning readiness assessment for Iraqi universities. 10th International and the 16th National Conference on E-Learning and E-Teaching, ICeLeT 2023. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICeLeT58996.2023.10139868
  4. Auri Pramesti, A., Sopiya, N., & Panigor Sitompul, R. (2022). Systematic Literature Review: Pemanfaatan Virtual Reality (VR) Sebagai Alternatif Media Pembelajaran. Jurnal Pendidikan Teknologi Dan Kejuruan, 19(2)
  5. Ependi, U., Putra, A., & Panjaitan, F. (2019). Evaluasi tingkat kebergunaan aplikasi administrasi penduduk menggunakan teknik system usability scale. Register: Jurnal Ilmiah Teknologi Sistem Informasi, 5(1), 63–76. https://doi.org/10.26594/register.v5i1.1412
  6. Guo, M., & Peng, F. (2023). A Survey of the Logistics Service Quality Level for Courier Enterprises based on Likert Scale using the R Language. Proceedings of 2023 IEEE 3rd International Conference on Information Technology, Big Data and Artificial Intelligence, ICIBA 2023, 1586–1589. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICIBA56860.2023.10164885
  7. Hariyanto, D., Triyono, M. B., & Köhler, T. (2020). Usability evaluation of personalized adaptive e-learning system using USE questionnaire. Knowledge Management and E-Learning, 12(1), 85–105. https://doi.org/10.34105/j.kmel.2020.12.005
  8. Iryanti, E., Ode, L., Zulfiqar, M., Kusumawardani, S., & Hidayah, I. (2022). Pengukuran Kepuasan Pengguna E-Learning Menggunakan Metode Evaluasi Heuristik Dan System Usability Scale. 9(3), 469–478. https://doi.org/10.25126/jtiik.202294631
  9. Khlaifat, N., Alnuman, N., & Al-Olamari, Z. (2024). Attitudes and Perspectives for Installing EV Charging Station: A Case Study at Isra University, Jordan. 2024 22nd International Conference on Research and Education in Mechatronics, REM 2024, 80–85. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/REM63063.2024.10735463
  10. Kusumaningtyas, A., & Prihandoko, P. (2024). Evaluasi Layanan Kesehatan Aplikasi Depok Single Window Dengan Metode System Usability Scale dan Heuristic Evaluation. Jurnal Teknologi Informasi Dan Ilmu Komputer, 11(1), 167–174. https://doi.org/10.25126/jtiik.20241117714
  11. Malioy, R. J., Sediyono, E., & Maria, E. (2024). Evaluasi Kepuasan Pengguna Website Menggunakan Framework PIECES: Studi Pada Pemerintah Kabupaten Maluku Barat Daya. Jurnal Sistem Informasi Bisnis, 14(2), 171–180. https://doi.org/10.21456/vol14iss2pp171-180
  12. Masruroh, S. U., Rizqullah, M. A. A., Khairani, D., Sukmana, H. T., Aripiyanto, S., & Putri, R. A. (2024). UI/UX Application of Bus Rapid Transit (BRT) System using User-Centered Design Method and Think Aloud Evaluation. 2024 3rd International Conference on Creative Communication and Innovative Technology, ICCIT 2024. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICCIT62134.2024.10701192
  13. Mendoza Diaz, N. V., Yoon, S. Y., Trytten, D. A., & Meier, R. (2023). Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students. IEEE Access, 11, 133099–133114. https://doi.org/10.1109/ACCESS.2023.3335931
  14. Mumtahana, H. A., & Pamungkas, R. (2022). Architecture Design Development of e-Learning for Primary School Learning in Madiun City. INTENSIF: Jurnal Ilmiah Penelitian Dan Penerapan Teknologi Sistem Informasi, 6(2), 285–292. https://doi.org/10.29407/intensif.v6i2.18276
  15. Novianti Puspitasari. (2019). 2019 International Conference on Electrical, Electronics and Information Engineering (ICEEIE). Samarinda: IEEE
  16. Octaviani, S., Triandini, E., & Hostiadi, D. P. (2025). Evaluating Lecturer Satisfaction on Academic Information System Using Usability and EUCS at Bandung University of Technology. MATRIK : Jurnal Manajemen, Teknik Informatika Dan Rekayasa Komputer, 25(1), 205–216. https://doi.org/10.30812/matrik.v25i1.4844
  17. Pangentasan, E. B., & Maria, E. (2023). Penerapan PIECES Framework dalam Analisis Kepuasan Pengguna F-Learn UKSW saat Pandemi Covid-19. J. Sistem Info. Bisnis, 13(1), 60–69. https://doi.org/10.21456/vol13iss1pp60-69
  18. Putera, W. A., & Candiasa, I. M. (2021). Analysis of e-learning user satisfaction itb stikom bali using end user computing satisfaction (eucs) method. Journal of Physics: Conference Series, 1810(1). IOP Publishing Ltd. https://doi.org/10.1088/1742-6596/1810/1/012017
  19. Saputri, M. E., Utami, F. N., & Sari, D. (2022). The Effectiveness of E-Learning Service Quality in Influencing E-Learning Student Satisfaction and Loyalty at Telkom University. Proceedings - International Conference Advancement in Data Science, E-Learning and Information Systems, ICADEIS 2022. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICADEIS56544.2022.10037454
  20. Setifani, N. A., Rolliawati, D., & Wahyudi, N. (2022). Analisis Pengaruh Digital Divide Terhadap User Satisfaction dan Individual Performance pada Pengguna Sistem Informasi Akademik. JURNAL SISTEM INFORMASI BISNIS, 12(1), 45–56. https://doi.org/10.21456/vol12iss1pp45-56
  21. Sun, Y. (2023). Investigation on Network Survey Questionnaire Evaluation Based on Neural Network Algorithm. 979–987. https://doi.org/10.1109/CICN.2023.99
  22. Syamsuar, D., Cholil, W., Ramadhan, R., Wadu, R. M. B., Zaidiah, A., & Oktaviani, N. (2022). Empirical Review Of M-Banking User Satisfaction Using End User Computing Satisfaction (EUCS). Proceedings - 4th International Conference on Informatics, Multimedia, Cyber and Information System, ICIMCIS 2022, 25–29. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICIMCIS56303.2022.10017742
  23. Tristan-Lopez, A. (2025). Enhancing Cronbach’s Alpha Applications through the Harmonised V-O-R Framework. Educational Methods and Psychometrics, 3(2025), 1–23. https://doi.org/10.61186/emp.2025.4
  24. Wibowo, J. A., Yasirandi, R., Utomo, R. G., & Pratama Anom, R. I. (2024). Comprehensive Evaluation of the Lakeside Café Point of Sale (POS) Application: A Synergistic Analysis Using SUS and EUCS Models. Proceedings - International Seminar on Intelligent Technology and Its Applications, ISITIA, (2024), 184–189. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ISITIA63062.2024.10667774
  25. Yudistira, A. R., Nuha, H. H., & Achmad, K. A. (2022). User Satisfaction Analysis of PeduliLindungi Application Using End User Computing Satisfaction (EUCS) Method
  26. Zhu, J., & Hou, H. (2021). Research on User Experience Evaluation of Mobile Applications in Government Services. IEEE Access, 9, 52634–52641. https://doi.org/10.1109/ACCESS.2021.3070365

Last update:

No citation recorded.

Last update: 2026-08-17 20:50:37

No citation recorded.