Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence.

Published

Journal Article

Despite a lack of face validity, there continues to be heavy reliance on objective paper-and-pencil measures of clinical competence. Among these measures, the most common item formats are patient management problems (PMPs) and three types of multiple choice questions (MCQs): one-best-answer (A-types); matching questions (M-types); and multiple true/false questions (X-types). The purpose of this study is to compare the reliability, validity and efficiency of these item formats with particular focus on whether MCQs and PMPs measure different aspects of clinical competence. Analyses revealed reliabilities of 0.72 or better for all item formats; the MCQ formats were most reliable. Similarly, efficiency analyses (reliability per unit of testing time) demonstrated the superiority of MCQs. Evidence for validity obtained through correlations of both programme directors' ratings and criterion group membership with item format scores also favoured MCQs. More important, however, is whether MCQs and PMPs measure the same or different aspects of clinical competence. Regression analyses of the scores on the validity measures (programme directors' ratings and criterion group membership) indicated that MCQs and PMPs seem to be measuring predominantly the same thing. MCQs contribute a small unique variance component over and above PMPs, while PMPs make the smallest unique contribution. As a whole, these results indicate that MCQs are more efficient, reliable and valid than PMPs.

Full Text

Duke Authors

Cited Authors

  • Norcini, JJ; Swanson, DB; Grosso, LJ; Webster, GD

Published Date

  • May 1, 1985

Published In

Volume / Issue

  • 19 / 3

Start / End Page

  • 238 - 247

PubMed ID

  • 4010571

Pubmed Central ID

  • 4010571

International Standard Serial Number (ISSN)

  • 0308-0110

Language

  • eng

Conference Location

  • England