Interpreting incremental value of markers added to risk prediction models.

Journal Article (Journal Article)

The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its insensitivity in model comparisons in which the baseline model has performed well. Thus, 2 other measures have been proposed to capture improvement in discrimination for nested models: the integrated discrimination improvement and the continuous net reclassification improvement. In the present study, the authors use mathematical relations and numerical simulations to quantify the improvement in discrimination offered by candidate markers of different strengths as measured by their effect sizes. They demonstrate that the increase in the AUC depends on the strength of the baseline model, which is true to a lesser degree for the integrated discrimination improvement. On the other hand, the continuous net reclassification improvement depends only on the effect size of the candidate variable and its correlation with other predictors. These measures are illustrated using the Framingham model for incident atrial fibrillation. The authors conclude that the increase in the AUC, integrated discrimination improvement, and net reclassification improvement offer complementary information and thus recommend reporting all 3 alongside measures characterizing the performance of the final model.

Full Text

Duke Authors

Cited Authors

  • Pencina, MJ; D'Agostino, RB; Pencina, KM; Janssens, ACJW; Greenland, P

Published Date

  • September 15, 2012

Published In

Volume / Issue

  • 176 / 6

Start / End Page

  • 473 - 481

PubMed ID

  • 22875755

Pubmed Central ID

  • PMC3530349

Electronic International Standard Serial Number (EISSN)

  • 1476-6256

Digital Object Identifier (DOI)

  • 10.1093/aje/kws207

Language

  • eng

Conference Location

  • United States