Skip to main content
construction release_alert
Scholars@Duke will be undergoing maintenance April 11-15. Some features may be unavailable during this time.
cancel
Journal cover image

How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.

Publication ,  Journal Article
Baker, SG; Schuit, E; Steyerberg, EW; Pencina, MJ; Vickers, A; Moons, KGM; Mol, BWJ; Lindeman, KS
Published in: Stat Med
September 28, 2014

An important question in the evaluation of an additional risk prediction marker is how to interpret a small increase in the area under the receiver operating characteristic curve (AUC). Many researchers believe that a change in AUC is a poor metric because it increases only slightly with the addition of a marker with a large odds ratio. Because it is not possible on purely statistical grounds to choose between the odds ratio and AUC, we invoke decision analysis, which incorporates costs and benefits. For example, a timely estimate of the risk of later non-elective operative delivery can help a woman in labor decide if she wants an early elective cesarean section to avoid greater complications from possible later non-elective operative delivery. A basic risk prediction model for later non-elective operative delivery involves only antepartum markers. Because adding intrapartum markers to this risk prediction model increases AUC by 0.02, we questioned whether this small improvement is worthwhile. A key decision-analytic quantity is the risk threshold, here the risk of later non-elective operative delivery at which a patient would be indifferent between an early elective cesarean section and usual care. For a range of risk thresholds, we found that an increase in the net benefit of risk prediction requires collecting intrapartum marker data on 68 to 124 women for every correct prediction of later non-elective operative delivery. Because data collection is non-invasive, this test tradeoff of 68 to 124 is clinically acceptable, indicating the value of adding intrapartum markers to the risk prediction model.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Stat Med

DOI

EISSN

1097-0258

Publication Date

September 28, 2014

Volume

33

Issue

22

Start / End Page

3946 / 3959

Location

England

Related Subject Headings

  • Statistics & Probability
  • Risk Assessment
  • ROC Curve
  • Pregnancy
  • Kaplan-Meier Estimate
  • Humans
  • Female
  • Decision Support Techniques
  • Cesarean Section
  • Area Under Curve
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Baker, S. G., Schuit, E., Steyerberg, E. W., Pencina, M. J., Vickers, A., Moons, K. G. M., … Lindeman, K. S. (2014). How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through. Stat Med, 33(22), 3946–3959. https://doi.org/10.1002/sim.6195
Baker, Stuart G., Ewoud Schuit, Ewout W. Steyerberg, Michael J. Pencina, Andrew Vickers, Karel G. M. Moons, Ben W. J. Mol, and Karen S. Lindeman. “How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.Stat Med 33, no. 22 (September 28, 2014): 3946–59. https://doi.org/10.1002/sim.6195.
Baker SG, Schuit E, Steyerberg EW, Pencina MJ, Vickers A, Moons KGM, et al. How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through. Stat Med. 2014 Sep 28;33(22):3946–59.
Baker, Stuart G., et al. “How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.Stat Med, vol. 33, no. 22, Sept. 2014, pp. 3946–59. Pubmed, doi:10.1002/sim.6195.
Baker SG, Schuit E, Steyerberg EW, Pencina MJ, Vickers A, Moons KGM, Mol BWJ, Lindeman KS. How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through. Stat Med. 2014 Sep 28;33(22):3946–3959.
Journal cover image

Published In

Stat Med

DOI

EISSN

1097-0258

Publication Date

September 28, 2014

Volume

33

Issue

22

Start / End Page

3946 / 3959

Location

England

Related Subject Headings

  • Statistics & Probability
  • Risk Assessment
  • ROC Curve
  • Pregnancy
  • Kaplan-Meier Estimate
  • Humans
  • Female
  • Decision Support Techniques
  • Cesarean Section
  • Area Under Curve