Adaptation and Validation of the Utrecht Work Engagement Scale-Student with Artificial Intelligence (UWES-S9AI) in Medical Students
DOI:
https://doi.org/10.69653/msth20234Keywords:
artificial intelligence - AI, academic commitment, games, validation, UWES-S9AIAbstract
Background: Academic engagement has been recognized as a pivotal component in the training of students, characterized by absorption, dedication, and vigor towards their studies. In the digital age, where Artificial Intelligence (AI) plays a central role in education, it is imperative to understand how students interact with and are engaged by these technological tools. Although scales measuring academic engagement exist, such as the Utrecht Work Engagement Scale (UWES-S and UWES-9S), there is a gap in the literature concerning their adaptation and validation in the context of AI. Objective: To adapt and validate the UWES-S9 Scale within the context of AI-related academic engagement, specifically among medical students. Methods: An instrumental study was conducted involving 479 medical students from two Peruvian universities aged between 18 and 37 years (M=20.22; SD=3.59). An AI-adapted version of the UWES-S9, termed UWES-S9AI, was utilized. A factorial analysis was performed to validate the scale's structure, along with an invariance analysis based on gender. Results: The analysis revealed a three-dimensional structure consistent with the original version of the UWES-S9. Reliability for all dimensions was adequate, and the scale's invariance concerning gender was also confirmed. Conclusions: The UWES-S9AI is a valid and reliable instrument to assess academic engagement in relation to AI among medical students. It explores academic commitment through the use of AI in educational contexts. However, it is essential to continue examining and adapting the tool in different settings and populations to ensure its universal applicability.
References
Abbas, N., Whitfield, J., Atwell, E., Bowman, H., Pickard, T., & Walker, A. (2022). Online chat and chatbots to enhance mature student engagement in higher education. International Journal of Lifelong Education, 41(3). https://doi.org/10.1080/02601370.2022.2066213
Ato, M., López, J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicología, 29(3), 1038–1059. https://doi.org/10.6018/analesps.29.3.178511
Bakker, A. B., Albrecht, S. L., & Leiter, M. P. (2011). Key questions regarding work engagement. European Journal of Work and Organizational Psychology, 20(1). https://doi.org/10.1080/1359432X.2010.485352
Balducci, C., Fraccaroli, F., & Schaufeli, W. B. (2010). Psychometric properties of the Italian version of the Utrecht Work Engagement Scale (UWES-9): A cross-cultural analysis. European Journal of Psychological Assessment, 26(2). https://doi.org/10.1027/1015-5759/a000020
Bandalos, D. L., & Finney, S. J. (2019). Factor analysis: Exploratory and confi rmatory. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (2da edició, pp. 98–120). Routledge.
Bohr, A., & Memarzadeh, K. (2020). The rise of artificial intelligence in healthcare applications. In Artificial Intelligence in Healthcare. https://doi.org/10.1016/B978-0-12-818438-7.00002-2
Brint, S., Cantwell, A. M., & Hanneman, R. A. (2008). The two cultures of undergraduate academic engagement. Research in Higher Education, 49(5). https://doi.org/10.1007/s11162-008-9090-y
Carmona-Halty, M. A., Schaufeli, W. B., & Salanova, M. (2019). The Utrecht work engagement scale for students (UWES-9S): Factorial validity, reliability, and measurement invariance in a chilean sample of undergraduate university students. Frontiers in Psychology, 10(APR), 1017. https://doi.org/10.3389/FPSYG.2019.01017/XML/NLM
Chen, F. F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
Chi, L. C., Tang, T. C., & Tang, E. (2022). Psychometric properties of the Utrecht Work Engagement Scale for Students (UWES-S) in the Taiwanese context. Current Psychology. https://doi.org/10.1007/s12144-022-03737-0
DiPerna, J. C., Volpe, R. J., & Elliott, S. N. (2002). A model of academic enablers and elementary reading/language arts achievement. In School Psychology Review (Vol. 31, Issue 3). https://doi.org/10.1080/02796015.2002.12086157
Dominguez Lara, S., Sánchez Villena, A. R., & Fernández Arata, M. (2020). Propiedades psicométricas de la UWES-9S en estudiantes universitarios peruanos. Acta Colombiana de Psicologia, 23(2), 24–39. https://doi.org/10.14718/ACP.2020.23.2.2
Escotet, M. Á. (2023). The optimistic future of Artificial Intelligence in higher education. Prospects. https://doi.org/10.1007/s11125-023-09642-z
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. In Review of Educational Research (Vol. 74, Issue 1). https://doi.org/10.3102/00346543074001059
Grassini, S. (2023). Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings. Education Sciences, 13(7), 692. https://doi.org/10.3390/educsci13070692
Greenwood, C. R., Delquadri, J., & Hall, R. V. (1984). Opportunity to respond and student academic performance. In J. Heward, W. Heron, T. Hill, D. TrapPorter (Ed.), Behavior analysis in education (pp. 58–88).
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2021). semTools: Useful tools for structural equation modeling. In The Comprehensive R Archive Network.
Kline, R. B. (2011). Principles and Practice of Structural Equation Modeling. Guilford Press.
Kline, R. B. (2016). Principles and practice of structural equation modeling (Cuarta Ed.). Guilford Press.
Loscalzo, Y., & Giannini, M. (2019). Study Engagement in Italian University Students: A Confirmatory Factor Analysis of the Utrecht Work Engagement Scale—Student Version. Social Indicators Research, 142(2). https://doi.org/10.1007/s11205-018-1943-y
Lovakov, A. V., Agadullina, E. R., & Schaufeli, W. B. (2017). Psychometric properties of the Russian version of the Utrecht work engagement scale (UWES-9). Psychology in Russia: State of the Art, 10(1). https://doi.org/10.11621/pir.2017.0111
Mangaroska, K., Martinez-Maldonado, R., Vesin, B., & Gašević, D. (2021). Challenges and opportunities of multimodal data in human learning: The computer science students’ perspective. Journal of Computer Assisted Learning, 37(4). https://doi.org/10.1111/jcal.12542
McDonald, R. P. (1999). Test Theory: A United Treatment. Lawrence Erlbaum.
Mir, M. M., Mir, G. M., Raina, N. T., Mir, S. M., Mir, S. M., Miskeen, E., Alharthi, M. H., & Alamri, M. M. S. (2023). Application of Artificial Intelligence in Medical Education: Current Scenario and Future Perspectives. In Journal of Advances in Medical Education and Professionalism (Vol. 11, Issue 3). https://doi.org/10.30476/jamp.2023.98655.1803
Muñiz, J., Elosua, P., & Hambleton, R. K. (2013). Directrices para la traducción y adaptación de los tests: Segunda edición. In Psicothema (Vol. 25, Issue 2). https://doi.org/10.7334/psicothema2013.24
Nations, U. (2020). Artificial intelligence and Inclusion: Compendium of promising initiatives Mobile Learning Week 2020. In International Law Reports.
Pérez, E. R., & Medrano, L. (2010). Análisis factorial exploratorio : Bases conceptuales y metodológicas. Revista Argentina de Ciencias Del Comportamiento, 2(1), 58–66.
Presidency of The Council of Ministers. (2023). Intelligence strategy national artificial. https://cdn.www.gob.pe/uploads/document/file/1909267/National Artificial Intelligence Strategy - Peru.pdf
Raykov, T., & Hancock, G. R. (2005). Examining change in maximal reliability for multiple-component measuring instruments. British Journal of Mathematical and Statistical Psychology, 58(1), 65–82. https://doi.org/10.1348/000711005X38753
Römer, J. (2016). The Korean utrecht work engagement scale-student (UWES-S): A factor validation study. TPM - Testing, Psychometrics, Methodology in Applied Psychology, 23(1). https://doi.org/10.4473/TPM23.1.5
Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48, 1–36. https://doi.org/10.18637/JSS.V048.I02
Sanchez-Cardona, I., Rodriguez-Montalban, R., Toro-Alfonso, J., & Velazquez, I. M. (2016). PSYCHOMETRIC PROPERTIES OF THE UTRECHT WORK ENGAGEMENT SCALE-STUDENT (UWES-S) IN UNIVERSITY STUDENTS IN PUERTO RICO. REVISTA MEXICANA DE PSICOLOGIA, 33(2).
Schaufeli, W. B., Bakker, A. B., & Salanova, M. (2006a). The measurement of work engagement with a short questionnaire: A cross-national study. Educational and Psychological Measurement, 66(4). https://doi.org/10.1177/0013164405282471
Schaufeli, W. B., Bakker, A. B., & Salanova, M. (2006b). The Measurement of Work Engagement With a Short Questionnaire. Educational and Psychological Measurement, 66(4), 701–716. https://doi.org/10.1177/0013164405282471
Schaufeli, W. B., Martínez, I. M., Pinto, A. M., Salanova, M., & Barker, A. B. (2002). Burnout and engagement in university students a cross-national study. Journal of Cross-Cultural Psychology, 33(5), 464–481. https://doi.org/10.1177/0022022102033005003
Schaufeli, W. B., Salanova, M., González-romá, V., & Bakker, A. B. (2002). The Measurement of Engagement and Burnout: A Two Sample Confirmatory Factor Analytic Approach. Journal of Happiness Studies 2002 3:1, 3(1), 71–92. https://doi.org/10.1023/A:1015630930326
Schaufeli, W. B., Shimazu, A., Hakanen, J., Salanova, M., & De Witte, H. (2019). An Ultra-Short Measure for Work Engagement. European Journal of Psychological Assessment, 35(4), 577–591. https://doi.org/10.1027/1015-5759/a000430
Schiff, D. (2021). Out of the laboratory and into the classroom: the future of artificial intelligence in education. AI and Society, 36(1). https://doi.org/10.1007/s00146-020-01033-8
Schumacker, R. E., & Lomax, R. G. (2016). A Beginner´s Guide to Structural Equation Modeling (4th ed.). Taylor & Francis.
Silva, J. O. M. da, Pereira Junior, G. A., Coelho, I. C. M. M., Picharski, G. L., & Zagonel, I. P. S. (2018). Engajamento entre Estudantes do Ensino Superior nas Ciências da Saúde (Validação do Questionário Ultrecht Work Engagement Scale (UWES-S) com Estudantes do Ensino Superior nas Ciências da Saúde). Revista Brasileira de Educação Médica, 42(2). https://doi.org/10.1590/1981-52712015v42n2rb20170112
Varma, J. R., Fernando, S., Ting, B. Y., Aamir, S., & Sivaprakasam, R. (2023). The Global Use of Artificial Intelligence in the Undergraduate Medical Curriculum: A Systematic Review. Cureus. https://doi.org/10.7759/cureus.39701
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J. B., Yuan, J., & Li, Y. (2021). A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. In Complexity (Vol. 2021). https://doi.org/10.1155/2021/8812542
