Suwarni Wijaya Halim
2016 Frans Seda Foundation Twin Scholarship Awardee
Thesis: Meaning-oriented assessment: an analysis of its inter-rater reliability, strengths, and weaknesses
ABSTRACT
This research looked into the implementation of Kim’s model of meaning-oriented assessment (2009) in assessing students’ translation works. The motivation behind this topic is the fact that there has been lack of research regarding the topic of translation assessment in the area of translation pedagogy, and there has been almost no previous research about the usage of meaning-oriented assessment in Indonesian context. This research attempted to fill the gap and aims at discovering the result of assessment using meaning-oriented assessment model, determining the interreliability of the model, and finding out the strengths and weaknesses of the model. To achieve the aims of the research, the researcher utilized mixed method with triangulation as the research design. Selected students were asked to translate an editorial taken from newspaper from English to Indonesian, and the results of their translation were assessed by three raters using meaning-oriented assessment. The assessment sheets were observed and compared, and the scores given by the raters were inputted into SPSS in order to prove whether or not the model has inter-rater reliability. Afterwards, interview towards the raters were conducted in order to find out the strengths and weaknesses of the model. The result of the analysis shows that meaning-oriented assessment either had low level of inter-rater reliability or no interrater reliability, and according to the results of interview, the researcher found that the raters perceived the detailed criteria and the high validity as the strengths of meaning oriented assessment. Nevertheless, they also thought that meaning-oriented assessment is too time-consuming and complicated. In addition, they believed meaning-oriented assessment might not produce reliable results, and its implementation might be problematic, both for formative and summative assessment. As a final remark, the researcher believed that the implementation of meaningoriented assessment would require great effort from the translator trainers, so the researcher suggested modification or proper and sufficient training process for any parties who would like to implement meaning-oriented assessment.
Keywords: meaning-oriented translation assessment, translation pedagogy