How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.

Published

Journal Article

An important question in the evaluation of an additional risk prediction marker is how to interpret a small increase in the area under the receiver operating characteristic curve (AUC). Many researchers believe that a change in AUC is a poor metric because it increases only slightly with the addition of a marker with a large odds ratio. Because it is not possible on purely statistical grounds to choose between the odds ratio and AUC, we invoke decision analysis, which incorporates costs and benefits. For example, a timely estimate of the risk of later non-elective operative delivery can help a woman in labor decide if she wants an early elective cesarean section to avoid greater complications from possible later non-elective operative delivery. A basic risk prediction model for later non-elective operative delivery involves only antepartum markers. Because adding intrapartum markers to this risk prediction model increases AUC by 0.02, we questioned whether this small improvement is worthwhile. A key decision-analytic quantity is the risk threshold, here the risk of later non-elective operative delivery at which a patient would be indifferent between an early elective cesarean section and usual care. For a range of risk thresholds, we found that an increase in the net benefit of risk prediction requires collecting intrapartum marker data on 68 to 124 women for every correct prediction of later non-elective operative delivery. Because data collection is non-invasive, this test tradeoff of 68 to 124 is clinically acceptable, indicating the value of adding intrapartum markers to the risk prediction model.

Full Text

Duke Authors

Cited Authors

  • Baker, SG; Schuit, E; Steyerberg, EW; Pencina, MJ; Vickers, A; Moons, KGM; Mol, BWJ; Lindeman, KS

Published Date

  • September 2014

Published In

Volume / Issue

  • 33 / 22

Start / End Page

  • 3946 - 3959

PubMed ID

  • 24825728

Pubmed Central ID

  • 24825728

Electronic International Standard Serial Number (EISSN)

  • 1097-0258

International Standard Serial Number (ISSN)

  • 0277-6715

Digital Object Identifier (DOI)

  • 10.1002/sim.6195

Language

  • eng