Investigating the Accuracy of GPT’s Translation in Transferring Syntax-Semantic Interface: A Case Study of Ideational Meta-Functions in English Translation of Quran

Farokhipour, Sajjad; Padiz, Davoud; Hekmatshoartabari, Bamshad

doi:10.22059/jolr.2025.391037.666912

Document Type : Research Paper

Authors

¹ Department of Language and Literature, Shahid Mahallati University of Islamic Sceinces, Qom, Iran.

² Department of English Language and Literature, Fculty of Hunamities, Yasuj University

³ Department of English Language and Literature, Faculty of Humanities, Yasuj University

https://doi.org/10.22059/jolr.2025.391037.666912

Abstract

AI-based models of analyzing linguistic data have attracted researchers due to the theoretical and practical spread of these models in recent years. Among them, machine translation and AI-based translation have been a focal point in this development However, many questions regarding their efficiency and accuracy are raised at the same time. The current research is an attempt to employ an evaluative framework from systemic functional linguistics and investigate the accuracy of the transfer of ideational meta-function in the English translation of Quranic verses by GPT-4o. To this aim, 200 verses of the holy Quran were selected purposively and the transfer of six categories of the aforesaid meta-functions including material, verbal, existential, mental, behavioral, and existential were explored based on verb category, participants, and conditions of each process. Results revealed that GPT has been able to transfer material, behavioral, and existential processes while failing to transfer mental and relational processes. It seems that GPT’s success in the first section cannot be attributed to AI’s ability to distinguish the qualities and conditions of the processes but to the limitedness of verbal category and objectivity of their semantic components, participant types, and conditions. Since the characteristics of each part of speech for each process are determined and defined in systemic functional linguistics, they can be attached to the machine translation models in the form of syntactic-semantic tags added to each lexical-grammatical unit to increase the accuracy of translation.

Keywords

Main Subjects

Click subject name to see related articles.

References

Abdelaal, N. M., & Rashid, S. M. (2016). Grammar-related semantic losses in the translation of the Holy Quran, with special reference to Surah Al A’araf (the heights). SAGE Open, 6(3), 1-11. https://doi.org/10.1177/2158244016661750.

Abdelaal, N. M. (2019). Faithfulness in the Translation of the Holy Quran: Revisiting the Skopos Theory. SAGE Open, 9(3), 1-14. https://doi.org/10.1177/2158244019873013.

Ali Mohsen,M., Althebi, S & Albahooth, M. (2023). A scientometric study of three decades of machine translation research: Trending issues, hotspot research, and co-citation analysis. Cogent Arts & Humanities, 10(1). https://doi.org/10.1080/23311983.2023.2242620.

Al-Qahtani, O.A. (2018). Investigating the translation of euphemism in the Quran from Arabic into English [Unpublished PhD dissertation]. Liverpool John Moors University.

Amplayo, R. K., K. M. Yoo, and S., W. Lee. (2022). Attribute injection for pre-trained language models: A new benchmark and an efficient method. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1051–1064, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Awad, A. (2005). Translating Arabic into English with special reference to Quranic Discourse [Unpublished PhD Dissertation]. University of Manchester.

Badr, H. and Menacere, K. (2019). Assessing the Translation Quality of Quranic Collocations: For better or for worse. International Journal of Linguistics, Literature and Translation 2 (2). 72-98. http// doi:10.32996/ijllt.2019.2.2.15.

Baker, M. (2011). In other words: a course book on translation, Routledge.

Brown, T., B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165.

Catford, J.C. (1965). A linguistic Theory of Translation. Oxford University Press.

Chesterman, A. (1997). Memes of Translation. John Benjamins.

Chowdhery,A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.-M., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Noam Shazeer, Prabhakaran, V., & Reif, E. (2022). PaLM: Scaling Language Modeling with Pathways. ArXiv (Cornell University). https://doi.org/10.48550/arxiv.2204.02311.

Derewianka, B., & Jones, P. (2016). Teaching language in context. Oxford University Press.

Eggins, S. (2004). Introduction to systemic functional linguistics. A&C Black.

Farghal, M., & Haider, A. S. (2024). Translating classical Arabic verse: human translation vs. AI large language models (Gemini and ChatGPT). Cogent Social Sciences, 10(1). https://doi.org/10.1080/23311886.2024.2410998.

Farrokhipour, S. (2023). Translation Quality from Systemic Functional Perspective: The Case of Ideational Meta-Functions in Quranic Translated Samples. Iranian Journal of Translation Studies, 21(83). https://journal.translationstudies.ir/ts/article/view/1123.

Feng, X., Z. Feng, W. Zhao, B. Qin, and T. Liu. 2020. Enhanced neural machine translation by joint decoding with word and pos-tagging sequences. Mobile Networks and Applications, 25(5):1722–1728.

Fowler, R. (1981). Literature as a Social Discourse: The Practice of Linguistic Criticism, Batsford.

Gebhard, M., & Accurso, K. (2020). Systemic functional linguistics. In C. Chapelle (Ed.), the Concise Encyclopedia of Applied Linguistics (pp. 1029–1037).

Goyal, N., C. Gao, V. Chaudhary, P.-J. Chen, G. Wenzek, D. Ju, S. Krishnan, M. Ranzato, F. Guzmán, and A. Fan. (2022). The flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538.

Halliday, M. A. K., & Matthiessen, C. M. I. M. (2014). Halliday’s introduction to functional grammar. Routledge.

Halliday, M.A.K. and Hasan, R. (1985). Language, Context and Text: Aspects of Language in a Social-Semiotic Perspective. Deakin University Press.

Halliday, M. A. K. (1993). Towards a language-based theory of learning. Linguistics and Education, 5(2), 93–116. https://doi.org/10.1016/0898-5898(93)90026-7.

Hassan, H.B, (2019). Investigating the challenges of translating Arabic collocations into English with reference to the Quran [Unpublished PhD Dissertation], John Moores University.

Hatim, B. & Munday, J. (2004). Translation: an advanced resource book, Routledge.

Hlaing, Z. Z., Thu, Y. K., Supnithi, T., & Netisopakul, P. (2022). Improving neural machine translation with POS-tag features for low-resource language pairs. Heliyon, 8(8), e10375. https://doi.org/10.1016/j.heliyon.2022.e10375.

Jiang, E., E. Toh, A. Molina, K. Olson, C. Kayacik, A. Donsbach, C. J. Cai, and M. Terry. (2022). Discovering the syntax and strategies of natural language programming with generative language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA. Association for Computing Machinery.

Jiao, H., Peng, B., Zong, L., Zhang, X., & Li, X. (2024). Gradable Chat GPT Translation Evaluation. ArXiv.org. https://arxiv.org/abs/2401.09984.

Karmaker, S. K. and D. Feng. 2023. Teler: A general taxonomy of LLM prompts for benchmarking complex tasks. arXiv preprint ar-Xiv:2305.11430.

Killean, C. (2004). Arabic Encyclopedia of the Modern Middle East and North Africa. Available at: http://www.encyclopedia.com/doc/1G2-3424600277.html.

Liu, S., & Zhu, W. (2023). An Analysis of the Evaluation of the Translation Quality of Neural Machine Translation Application Systems. Applied Artificial Intelligence, 37(1). https://doi.org/10.1080/08839514.2023.2214460.

Mallah, F.E. (2008). Arabic English Translational Crossover Viewed from a Linguistic/ Cultural Perspective [Unpublished Ph.D. Dissertation]. Edinburgh University.

Mehri, A., Farokhipour, S., Sajjadi Dehkharghani, S. (2020). On the Impact of Global Coherence on Translation Comprehension of the Holy Quran: A Case Study of PhD-ESP Learners. Linguistic Research in the Holy Quran, 9(2), 31-36.

Najjar, I. I. (2020). Quranic stylistic shift in translation: with reference to conjunctive particle shift. Journal of Intercultural Communication Research, 49(6), 585–599.

Newmark, P. (1988). A textbook of translation. Pearson Education.

Nida, E. A., & Taber, C. R. (1974). The theory and practice of translation. E.J. Brill.

Öner Bulut,S., & Alimen., N. (2023). Translator education as a collaborative quest for insights into the re-positioning of the human translator (educator) in the age of machine translation: the results of a learning experiment. Interpreter and Translator Trainer, 17(3), 375–392. https://doi.org/10.1080/1750399x.2023.2237837.

Pan, W., Q. Chen, X. Xu, W. Che, and L. Qin. (2023). A preliminary evaluation of CHAT-GPT for zero-shot dialogue understanding. arXiv preprint arXiv:2304.04256.

Pickthail, M.M. (2006). Translation of the holy Quran. Retrieved from www.alquran.info.

Popescu-Belis, A. (2019). Context in neural machine translation: A review of models and evaluations. arXiv preprint arXiv:1901.09115.

Pym, A. (2018). Translation and text transfer. Intercultural Studies Group.

Sarani, A. and Farokhipour, S. (2017). The Effect of Genre Awareness on English Translation Quality and Pedagogy: A Case of News Reports Translation as an Academic Curriculum. Iranian Journal of Applied Language Studies, 9 (Proceedings of the First International Conference on Language Focus), 187-208. doi: 10.22111/ijals.2017.4293.

Shehab, E., & Daragmeh, A. (2014). A Context-based Approach to Proverb Translation: The Case of Arabic into English Translation. Translation Review, 90(1), 51–68. https://doi.org/10.1080/07374836.2014.986779.

Touvron, H., T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.

Vinay, J. P., & Darbelnet, J. (1995). Comparative stylistics of French and English: A methodology for translation. John Benjamins Publishing.

Wendland, E. R. (2012). Framing the frames: A theoretical framework for the cognitive notion of “frames of reference. Journal of Translation, 6, 27-50. https://doi.org/10.54395/jot-ww9v6.

Language Research

Investigating the Accuracy of GPT’s Translation in Transferring Syntax-Semantic Interface: A Case Study of Ideational Meta-Functions in English Translation of Quran

References

References

Volume 16, Issue 1 - Serial Number 30
November 2025
Pages 141-166

Investigating the Accuracy of GPT’s Translation in Transferring Syntax-Semantic Interface: A Case Study of Ideational Meta-Functions in English Translation of Quran

References

References

Volume 16, Issue 1 - Serial Number 30November 2025Pages 141-166

Volume 16, Issue 1 - Serial Number 30
November 2025
Pages 141-166