Multiple Teacher Ratings: An evaluation of measurement strategies.

Published

Journal Article

This study addresses the questions that arise when collecting, describing, and analyzing information from multiple informants regarding attributes of individual students. Using data from the Fast Track study, we evaluate alternative measurement strategies for using multiple teacher ratings of student adjustment to middle school among a sample of 326 Grade-6 pupils. One goal of the study was to compare the advantages of three measurement strategies using multiple and single informants in terms of their correlation with contemporaneous measures of behavior and academic achievement. Comparisons of residual variance using an aggregated rating, the rating from an "optimal informant," and a score selected at random from the response set, indicate that aggregation provides the highest criterion-related validity. As part of these analyses, we explore the significance of inter-rater concordance, measured in terms of the intraclass correlation coefficient (ICC). Results indicate that for some aggregated scores, reliability can significantly limit their interpretability. The second main goal of the study was to evaluate the effects of variation in die number of teacher ratings on residual variance estimates for aggregate measures in selected behavioral domains. We conclude that the advantages of using multiple ratings are significant with a larger number of informants.

Full Text

Duke Authors

Cited Authors

  • Muschkin, CG; Malone, PS; Conduct Problems Prevention Research Group,

Published Date

  • February 2007

Published In

Volume / Issue

  • 13 / 1

Start / End Page

  • 71 -

PubMed ID

  • 20098624

Pubmed Central ID

  • 20098624

Electronic International Standard Serial Number (EISSN)

  • 1744-4187

International Standard Serial Number (ISSN)

  • 1380-3611

Digital Object Identifier (DOI)

  • 10.1080/13803610601058215

Language

  • eng