A Comparative Study of EFL Instructors’ Essay Rating: Holistic Versus Analytic Approaches at a Tertiary Institution in Saudi Arabia
DOI:
https://doi.org/10.17507/tpls.1201.07Keywords:
critical language testing, EFL writing, holistic and analytic rubrics, Saudi EFL contextAbstract
This study seeks to explore the factors that influence EFL instructors’ rating decisions while using holistic and analytic rubrics. Few studies have been conducted on the factors that influence the rating practices of EFL instructors, specifically, in the Saudi EFL context. This study addresses this gap and contributes more broadly to the understanding of the interplay between EFL instructors and the use of holistic and analytic rubrics. The data were collected in a Saudi university at a preparatory year program (PYP). This study utilizes semi-structured interviews with eleven EFL instructors from different nationalities. Guided by the critical language testing as a theoretical framework and with qualitative analysis, the study reveals that critical language testing can minimize the negative consequences of writing assessment done by graders; however, students’ low English proficiency, time constraints, heavy workload can negatively affect the rating practices. Finally, several pedagogical implications, insights, and recommendations for future research are put forward in the conclusion.
References
Al-Seghayer, K. (2005). Teaching English in the kingdom of Saudi Arabia: Slowly but steadily changing. In G. Braine (Ed), Teaching English to the world: History, curriculum, and practice (pp. 125-134). Mahwah, NJ: Lawrence Erlbaum.
Aldukhayel, D. M. (2017). Exploring students’ perspectives toward clarity and familiarity of writing scoring rubrics: The case of Saudi EFL students. English Language Teaching, 10(10), 1-9.
Alsamaani, A. (2014). Evaluating classroom assessment techniques of novice Saudi EFL teachers. Journal of Arabic and Human Sciences, 270(1687), 1-40.
Ansari, A. A. (2012). Teaching of English to Arab students: Problems and remedies. Educational Research, 3(6), 519-524.
Bacha, N. (2001). Writing evaluation: what can analytic versus holistic essay scoring tell us? System, 29(3), 371-383.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Barkaoui, K. (2010). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54-74.
Burkhart, F. (1999). Samuel Messick, 67, leader in the educational testing field. European Journal of Psychological Assessment, 15(1), 87-87.
Dörnyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University Press.
Ghalib, T. K., & Al-Hattami, A. A. (2015). Holistic versus Analytic Evaluation of EFL Writing: A Case Study. English Language Teaching, 8(7), 225-236.
Graham, B. (2000). Trends & Issues in Postsecondary English Studies. National Council of Teachers of English, 1111 W. Kenyon Road, Urbana, IL 61801-1096.
Hamouda, A. (2011). A Study of students and teachers’ preferences and attitudes towards correction of classroom written errors in Saudi EFL context. English Language Teaching, 4(3), 128-141.
Hamp-Lyons, L. (2002). The scope of writing assessment. Assessing writing, 8(1), 5-16.
Han, T., & Huang, J. (2017). Examining the impact of scoring methods on the institutional EFL writing assessment: A Turkish perspective. PASAA: Journal of Language Teaching and Learning in Thailand, 53, 112-147.
Hussein, A., & Mohammad, M. (2011). Negative L1 impact on L2 writing. International Journal of Humanities and Social Science, 1(18), 184-195.
Javid, C., & Umer, M. (2014). Saudi EFL learners’ writing problems: a move towards solution. Proceeding of the Global Summit on Education GSE, 4-5.
Kayapınar, U. (2014). Measuring essay assessment: Intra-rater and inter-rater reliability. Eurasian Journal of Educational Research, (57), 113-136.
Kunnan, A. J. (2004). Test fairness. In Milanovic, M. & Weir, C. (Eds). European language testing in a global context: Proceedings of the ALTE Barcelona Conference (pp. 27-48). Cambridge University Press.
Lim, G. S. (2011). The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced raters. Language Testing, 28(4), 543-560.
Mahshanian, A., Ketabi, S., & Abbas, R. (2017). Raters’ fatigue and their comments during scoring writing essays: A case of Iranian EFL learners. Indonesian Journal of Applied Linguistics, 7(2), 302-314.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50: 741–9.
Messick, S. (1996). Validity and washback in language testing. Language testing, 13(3), 241-256.
Mohammad, T., & Hazarika, Z. (2016). Difficulties of learning EFL in KSA: Writing skills in context. International Journal of English Linguistics, 6(3), 105-117.
Obeid, R. (2017). Second Language Writing and Assessment: Voices from within the Saudi EFL Context. English Language Teaching, 10(6), 174-181.
Richardson, P. M. (2004). Possible influences of Arabic-Islamic culture on the reflective practices proposed for an education degree at the Higher Colleges of Technology in the United Arab Emirates. International Journal of Educational Development, 24, 429-436.
Schaefer, E. (2008). Rater bias patterns in an EFL writing assessment. Language Testing, 25(4), 465-493.
Shohamy, E. (1998). Critical language testing and beyond. Studies in educational evaluation, 24(4), 331-45.
Shukri, N. A. (2014). Second language writing and culture: Issues and challenges from the Saudi learners’ perspective. Arab World English Journal, 5(3), 190-207.
Yancey, K. B. (1999). Looking back as we look forward: Historicizing writing assessment. College composition and communication, 50(3), 483-503.
