VALIDITY AND RELIABILITY OF A GENERATIVE ARTIFICIAL INTELLIGENCE (GEN AI) INSTRUMENT FOR ARABIC LANGUAGE STUDENTS AT PUBLIC UNIVERSITIES IN THE KLANG VALLEY: A PILOT STUDY

Mohammad Najib   Jaffar; Nurkhamimi Zainuddin; Muhammad Hakim Kamal; Mohammad Imran Ahmad

doi:10.64757/alqanatir.2026.3506/1522

Authors

Mohammad Najib Jaffar Faculty of Major Language Studies, Universiti Sains Islam Malaysia
Nurkhamimi Zainuddin Faculty of Major Language Studies, Universiti Sains Islam Malaysia
Muhammad Hakim Kamal Kolej Permata Insan, Universiti Sains Islam Malaysia
Mohammad Imran Ahmad Faculty of Islamic Civilization Studies, Universiti Islam Selangor

DOI:

https://doi.org/10.64757/alqanatir.2026.3506/1522

Keywords:

Arabic, Language, Learning, Generative Artificial Intelligence, Students

Abstract

This study examines the validity and reliability of a research instrument designed to investigate the use of Generative Artificial Intelligence (GenAI) among Arabic language students at public universities in the Klang Valley, Malaysia. Content validity was established through evaluation by three field experts using the Kappa coefficient analysis, yielding an overall Kappa value of 0.72, indicating a good level of agreement. Subsequently, a pilot study involving 36 respondents was conducted to assess the reliability of the instrument using Cronbach’s Alpha. All constructs, namely Lacks, Necessities, and Wants, demonstrated Cronbach’s Alpha values of 0.871, 0.954, and 0.957, respectively, confirming strong internal consistency. These findings confirm that the instrument is valid, reliable, and suitable for use in the main study.

Downloads

Download data is not yet available.

Author Biographies

Mohammad Najib Jaffar , Faculty of Major Language Studies, Universiti Sains Islam Malaysia

Associate Professor

Nurkhamimi Zainuddin, Faculty of Major Language Studies, Universiti Sains Islam Malaysia

Associate Professor

Muhammad Hakim Kamal, Kolej Permata Insan, Universiti Sains Islam Malaysia

Lecturer

Mohammad Imran Ahmad, Faculty of Islamic Civilization Studies, Universiti Islam Selangor

Senior Lecturer

References

Alayba, A. M. (2025). Arabic Natural Language Processing (NLP). Computers, 14(11), 497. https://doi.org/10.3390/computers14110497

Ali, M., & Omar, R. (2024). Overcoming Hurdles In Arabic Language Learning Through Genai Innovations. Journal of Modern Education.

Al-Mansouri, A., & Hassan, S. (2024). Challenges In Mastering Complex Arabic Grammar And The Role Of AI. Arabic Education Review.

Al-Onaizan, Y. (2024). Continuous Adaptation Of AI Systems To Evolving Linguistic Structures. AI & Language Research Journal.

Al-Shehri, S. (2024). Increasing Student Engagement In Arabic Language Learning Through GenAI. Digital Pedagogies in Language Learning.

Browne, R. H. (1995). On The Use Of A Pilot Sample For Sample Size Determination. Statistics in Medicine, 14(17), 1933–1940. https://doi.org/10.1002/sim.4780141709

Chan, K., & Lee, M. (2023). Ethical Guidelines For AI Usage In Higher Education: Preventing Academic Misconduct. Ethics and Technology.

Creswell, J. W., & Creswell, J. D. (2017). Research Design: Qualitative, Quantitative, And Mixed Methods Approaches (5th ed.). SAGE Publications.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate Data Analysis (8th ed.). Cengage Learning.

Hassan, S., Omar, R., & Malik, F. (2023). Lack Of Immersive Linguistic Environments In Public Universities. Malaysian Journal of Education.

Hutchinson, T., & Waters, A. (1987). English For Specific Purposes: A Learning-Centred Approach. Cambridge University Press.

Ismail, N., Rahman, A., Zulkifli, H., & Bakar, M. A. (2025). Student-Centred Pedagogical Approaches In Arabic Language Learning. Asian Education Review.

Jaffar, M. N., & Nazrol, N. A. H. (2025). The Use Of Generative Artificial Intelligence (Genai) Applications Among The Arabic Students: A Preliminary Study. Journal of Advanced Islamic and Legal Research, 2(2), 191–196.

Kamal, M. H. (2026a). Bibliometric Analysis Of Research On Gifted Education. Pertanika Journal of Social Sciences & Humanities, 34(1), 135–154. https://doi.org/10.47836/jssh.34.1.07

Kamal, M. H. (2026b). A Scoping Review Of Flipped Classroom Approaches In Arabic Teaching. Ijaz Arabi Journal of Arabic Learning, 9(1). https://doi.org/10.18860/ijazarabi.v9i1.36740

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., ... Kasneci, G. (2023). Chatgpt For Good? On Opportunities And Challenges Of Large Language Models For Education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

Khalil, M., & Er, E. (2023). Will Chatgpt Get You Caught? Rethinking Of Plagiarism Detection. arXiv. https://doi.org/10.48550/arXiv.2302.04335

Landis, J. R., & Koch, G. G. (1977). The Measurement Of Observer Agreement For Categorical Data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310

Lee, S., Choe, H., Zou, D., & Jeon, J. (2025). Generative AI (Genai) In The Language Classroom: A Systematic Review. Interactive Learning Environments. https://doi.org/10.1080/10494820.2025.2498537

Linur, R., Rusdi, Lubis, T., & Rohman, K. S. (2025). Student Perception Analysis Of AI's Impact On Arabic Language Learning: A Personality Perspective. Jurnal Al Bayan: Jurnal Jurusan Pendidikan Bahasa Arab, 17(2), 312–328. https://doi.org/10.24042/fya8h073

Moustafa, A. H. A., Al-Hamad, M. F., Qureshi, M. A., & Qadir, J. (2026). Generative AI For Learning And Teaching Arabic: Opportunities, Challenges, And Ethical Considerations. IEEE Access, 14. https://doi.org/10.1109/ACCESS.2026.3651864

Muhammad Hakim, K. (2026). Assessment Of Arabic Language Skills Using AI Tools Among Gifted Talented Students. Malaysian Journal of Social Sciences and Humanities (MJSSH), 11(1), e003761. https://doi.org/10.47405/mjssh.v11i1.3761

Okulicz-Kozaryn, W. (2026). Generative AI As A Student Research Assistant: The Relationship Of Academic And Research Practices In Higher Education. Sci, 8(1), Article 24. https://doi.org/10.3390/sci8010024

Saarela, M., Aljohani, N. R., Alhadlaq, A., Alowedi, N. A., & Aljohani, A. J. (2026). A Meta-Analysis Of Generative Artificial Intelligence Effects On Language Learning. Education And Information Technologies. Advance online publication. https://doi.org/10.1007/s10791-026-10015-1

Yaacob, M., & Abu Bakar, K. (2018). Student-Centered Teaching Approach Combined With Coaching And Mentoring Elements In Arabic Language Speaking Proficiency. Open Access Library Journal, 5, e4684. https://doi.org/10.4236/oalib.1104684

Yatri, D., Anugerahwati, M., & Setyowati, L. (2023). Artificial Intelligence (AI) In Language Learning (English And Arabic Class): Students' And Teachers' Experience And Perceptions. Transformational Language, Literature, and Technology Overview in Learning (TRANSTOOL), 3(1), 1–12. https://doi.org/10.55047/transtool.v3i1.1338

Yılmaz, R., & Karaoğlan Yılmaz, F. G. (2023). The Effect Of Generative Artificial Intelligence (AI)-Based Tool Use On Students' Computational Thinking Skills, Programming Self-Efficacy And Motivation. Computers and Education: Artificial Intelligence, 4, 100147. https://doi.org/10.1016/j.caeai.2023.100147

Yilmaz, R., & Yilmaz, F. (2023). Enhancing Academic Productivity Through GenAI Materials. Educational Technology Review.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic Review Of Research On Artificial Intelligence Applications In Higher Education—Where Are The Educators? International Journal of Educational Technology in Higher Education, 16, Article 39. https://doi.org/10.1186/s41239-019-0171-0