Skip to main content

Multiple Teacher Ratings: An evaluation of measurement strategies.

Publication ,  Journal Article
Muschkin, CG; Malone, PS; Conduct Problems Prevention Research Group,
Published in: Educational research and evaluation : an international journal on theory and practice
February 2007

This study addresses the questions that arise when collecting, describing, and analyzing information from multiple informants regarding attributes of individual students. Using data from the Fast Track study, we evaluate alternative measurement strategies for using multiple teacher ratings of student adjustment to middle school among a sample of 326 Grade-6 pupils. One goal of the study was to compare the advantages of three measurement strategies using multiple and single informants in terms of their correlation with contemporaneous measures of behavior and academic achievement. Comparisons of residual variance using an aggregated rating, the rating from an "optimal informant," and a score selected at random from the response set, indicate that aggregation provides the highest criterion-related validity. As part of these analyses, we explore the significance of inter-rater concordance, measured in terms of the intraclass correlation coefficient (ICC). Results indicate that for some aggregated scores, reliability can significantly limit their interpretability. The second main goal of the study was to evaluate the effects of variation in die number of teacher ratings on residual variance estimates for aggregate measures in selected behavioral domains. We conclude that the advantages of using multiple ratings are significant with a larger number of informants.

Duke Scholars

Published In

Educational research and evaluation : an international journal on theory and practice

DOI

EISSN

1744-4187

ISSN

1380-3611

Publication Date

February 2007

Volume

13

Issue

1

Start / End Page

71

Related Subject Headings

  • Education
  • 3904 Specialist studies in education
  • 3903 Education systems
  • 1303 Specialist Studies in Education
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Muschkin, C. G., Malone, P. S., & Conduct Problems Prevention Research Group, . (2007). Multiple Teacher Ratings: An evaluation of measurement strategies. Educational Research and Evaluation : An International Journal on Theory and Practice, 13(1), 71. https://doi.org/10.1080/13803610601058215
Muschkin, Clara G., Patrick S. Malone, and Patrick S. Conduct Problems Prevention Research Group. “Multiple Teacher Ratings: An evaluation of measurement strategies.Educational Research and Evaluation : An International Journal on Theory and Practice 13, no. 1 (February 2007): 71. https://doi.org/10.1080/13803610601058215.
Muschkin CG, Malone PS, Conduct Problems Prevention Research Group. Multiple Teacher Ratings: An evaluation of measurement strategies. Educational research and evaluation : an international journal on theory and practice. 2007 Feb;13(1):71.
Muschkin, Clara G., et al. “Multiple Teacher Ratings: An evaluation of measurement strategies.Educational Research and Evaluation : An International Journal on Theory and Practice, vol. 13, no. 1, Feb. 2007, p. 71. Epmc, doi:10.1080/13803610601058215.
Muschkin CG, Malone PS, Conduct Problems Prevention Research Group. Multiple Teacher Ratings: An evaluation of measurement strategies. Educational research and evaluation : an international journal on theory and practice. 2007 Feb;13(1):71.

Published In

Educational research and evaluation : an international journal on theory and practice

DOI

EISSN

1744-4187

ISSN

1380-3611

Publication Date

February 2007

Volume

13

Issue

1

Start / End Page

71

Related Subject Headings

  • Education
  • 3904 Specialist studies in education
  • 3903 Education systems
  • 1303 Specialist Studies in Education