Skip to main content
Journal cover image

Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.

Publication ,  Journal Article
Pencina, MJ; D'Agostino, RB; Vasan, RS
Published in: Stat Med
January 30, 2008

Identification of key factors associated with the risk of developing cardiovascular disease and quantification of this risk using multivariable prediction algorithms are among the major advances made in preventive cardiology and cardiovascular epidemiology in the 20th century. The ongoing discovery of new risk markers by scientists presents opportunities and challenges for statisticians and clinicians to evaluate these biomarkers and to develop new risk formulations that incorporate them. One of the key questions is how best to assess and quantify the improvement in risk prediction offered by these new models. Demonstration of a statistically significant association of a new biomarker with cardiovascular risk is not enough. Some researchers have advanced that the improvement in the area under the receiver-operating-characteristic curve (AUC) should be the main criterion, whereas others argue that better measures of performance of prediction models are needed. In this paper, we address this question by introducing two new measures, one based on integrated sensitivity and specificity and the other on reclassification tables. These new measures offer incremental information over the AUC. We discuss the properties of these new measures and contrast them with the AUC. We also develop simple asymptotic tests of significance. We illustrate the use of these measures with an example from the Framingham Heart Study. We propose that scientists consider these types of measures in addition to the AUC when assessing the performance of newer biomarkers.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Stat Med

DOI

ISSN

0277-6715

Publication Date

January 30, 2008

Volume

27

Issue

2

Start / End Page

157 / 172

Location

England

Related Subject Headings

  • United States
  • Statistics & Probability
  • Risk Factors
  • Risk Assessment
  • ROC Curve
  • Models, Statistical
  • Humans
  • Cardiovascular Diseases
  • Area Under Curve
  • 4905 Statistics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Pencina, M. J., D’Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med, 27(2), 157–172. https://doi.org/10.1002/sim.2929
Pencina, Michael J., Ralph B. D’Agostino, and Ramachandran S. Vasan. “Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.Stat Med 27, no. 2 (January 30, 2008): 157–72. https://doi.org/10.1002/sim.2929.
Pencina, Michael J., et al. “Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.Stat Med, vol. 27, no. 2, Jan. 2008, pp. 157–72. Pubmed, doi:10.1002/sim.2929.
Journal cover image

Published In

Stat Med

DOI

ISSN

0277-6715

Publication Date

January 30, 2008

Volume

27

Issue

2

Start / End Page

157 / 172

Location

England

Related Subject Headings

  • United States
  • Statistics & Probability
  • Risk Factors
  • Risk Assessment
  • ROC Curve
  • Models, Statistical
  • Humans
  • Cardiovascular Diseases
  • Area Under Curve
  • 4905 Statistics