Skip to main content
construction release_alert
Scholars@Duke will be undergoing maintenance April 11-15. Some features may be unavailable during this time.
cancel
Journal cover image

Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives.

Publication ,  Journal Article
Steyerberg, EW; Vedder, MM; Leening, MJG; Postmus, D; D'Agostino, RB; Van Calster, B; Pencina, MJ
Published in: Biom J
July 2015

New markers may improve prediction of diagnostic and prognostic outcomes. We aimed to review options for graphical display and summary measures to assess the predictive value of markers over standard, readily available predictors. We illustrated various approaches using previously published data on 3264 participants from the Framingham Heart Study, where 183 developed coronary heart disease (10-year risk 5.6%). We considered performance measures for the incremental value of adding HDL cholesterol to a prediction model. An initial assessment may consider statistical significance (HR = 0.65, 95% confidence interval 0.53 to 0.80; likelihood ratio p < 0.001), and distributions of predicted risks (densities or box plots) with various summary measures. A range of decision thresholds is considered in predictiveness and receiver operating characteristic curves, where the area under the curve (AUC) increased from 0.762 to 0.774 by adding HDL. We can furthermore focus on reclassification of participants with and without an event in a reclassification graph, with the continuous net reclassification improvement (NRI) as a summary measure. When we focus on one particular decision threshold, the changes in sensitivity and specificity are central. We propose a net reclassification risk graph, which allows us to focus on the number of reclassified persons and their event rates. Summary measures include the binary AUC, the two-category NRI, and decision analytic variants such as the net benefit (NB). Various graphs and summary measures can be used to assess the incremental predictive value of a marker. Important insights for impact on decision making are provided by a simple graph for the net reclassification risk.

Duke Scholars

Published In

Biom J

DOI

EISSN

1521-4036

Publication Date

July 2015

Volume

57

Issue

4

Start / End Page

556 / 570

Location

Germany

Related Subject Headings

  • Statistics & Probability
  • Risk
  • ROC Curve
  • Models, Statistical
  • Middle Aged
  • Humans
  • Decision Support Techniques
  • Coronary Disease
  • Computer Graphics
  • Cholesterol, HDL
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Steyerberg, E. W., Vedder, M. M., Leening, M. J. G., Postmus, D., D’Agostino, R. B., Van Calster, B., & Pencina, M. J. (2015). Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives. Biom J, 57(4), 556–570. https://doi.org/10.1002/bimj.201300260
Steyerberg, Ewout W., Moniek M. Vedder, Maarten J. G. Leening, Douwe Postmus, Ralph B. D’Agostino, Ben Van Calster, and Michael J. Pencina. “Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives.Biom J 57, no. 4 (July 2015): 556–70. https://doi.org/10.1002/bimj.201300260.
Steyerberg EW, Vedder MM, Leening MJG, Postmus D, D’Agostino RB, Van Calster B, et al. Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives. Biom J. 2015 Jul;57(4):556–70.
Steyerberg, Ewout W., et al. “Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives.Biom J, vol. 57, no. 4, July 2015, pp. 556–70. Pubmed, doi:10.1002/bimj.201300260.
Steyerberg EW, Vedder MM, Leening MJG, Postmus D, D’Agostino RB, Van Calster B, Pencina MJ. Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives. Biom J. 2015 Jul;57(4):556–570.
Journal cover image

Published In

Biom J

DOI

EISSN

1521-4036

Publication Date

July 2015

Volume

57

Issue

4

Start / End Page

556 / 570

Location

Germany

Related Subject Headings

  • Statistics & Probability
  • Risk
  • ROC Curve
  • Models, Statistical
  • Middle Aged
  • Humans
  • Decision Support Techniques
  • Coronary Disease
  • Computer Graphics
  • Cholesterol, HDL