Testing across languages in international comparisons: cultural adaptation of consensus-based test translation review procedures

We investigated whether consensus-based test translation review procedures can be used effectively in cultural contexts with high social stratification. We staged two test translation review panels in China–where social stratification may potentially inhibit individuals’ ability to express opinion and disagreement. We adapted a consensus-based test translation review procedure in ways intended to ensure that all panellists could express their thinking. With the adapted procedure, we facilitated two panels charged with identifying errors in Chinese translations of items from PISA (OECD’s international test on skills for life). While social interactions during the panels reflected social status differences, the adaptations made it possible to capture the thinking of all panellists. The panels identified similar numbers of translation errors, but different sets of translation errors. We conclude that cultural adaptation addressing social stratification is critical to successfully implementing consensus-based procedures for test translation review in contexts with high social stratification. Ultimately, it contributes to ensuring fair, valid testing for culturally and linguistically diverse populations.

Duke Scholars

Author Xueyu Zhao DKU Faculty