Zhu, Y., Fung, S.L., Tse, S.K. & Hsieh, C.Y. (2017). Exploring the Relationship Between Raters’ Personality Traits and Rating Severity in Writing Assessment of Chinese as a Second Language: A Pilot Study. In Zhang, D & Lin CH (Eds.), Chinese as a Second Language Assessment, p. 203-223. Singapore: Springer, 2017

第二語文學習, 寫作, 教學研究

Title

Exploring the Relationship Between Raters’ Personality Traits and Rating Severity in Writing Assessment of Chinese as a Second Language: A Pilot Study

Book

Chinese as a Second Language Assessment

Year

2017

Page

pp203-223

Author

Yu ZHU, Shui-Lung FUNG, Shek-Kam TSE, Chi-Yi HSIEH

Editor

Dongbo ZHANG, Chin-Hsi LIN

Publisher

Springer

Link / doi

Abstract

Subjective rating is a widely adopted practice in human assessment and measurement. Without a careful control of severity among raters in grading, both the validity and reliability of a subjective assessment will be subject to skepticism. Among many variables which may potentially impact raters’ severity, raters’ personality in recent years has caused concerns for many researchers. However, so far there is no study investigating such a potential relationship in the context of testing and assessing Chinese as a second language. To explore the relationship between raters’ personality traits and rating severity in writing assessment of Chinese as a foreign language, master’s degree students who majored in teaching of Chinese as a foreign language (n = 28) were asked to rate a writing test of the New HSK, a large-scale standardized test of Chinese proficiency promoted by the Chinese government. Valid data for FACETS analysis were the 28 raters’ independent ratings of 77 examinees’ 154 essays (2 topics) from the test. Each rater also completed NEO-PI-R, a fairly reliable personality inventory. Demographic variables such as age, gender, educational background, and rating experience, which may affect raters’ severity in essay grading, were strictly controlled. The results of this pilot study showed individual differences in raters’ severity and significant associations between raters’ personality traits and their rating behavior in terms of severity. Findings of this study are expected to enrich our understanding of subjective rating behaviors of human assessment, especially in the field of testing and assessing Chinese as a second language.