Skip to main content
construction release_alert
Scholars@Duke will be undergoing maintenance April 11-15. Some features may be unavailable during this time.
cancel
Journal cover image

Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration.

Publication ,  Journal Article
Pencina, MJ; Fine, JP; D'Agostino, RB
Published in: Stat Med
December 10, 2017

Discrimination slope, defined as the slope of a linear regression of predicted probabilities of event derived from a prognostic model on the binary event status, has recently gained popularity as a measure of model performance. It is as a building block for the integrated discrimination improvement that equals the difference in discrimination slopes between the two models being compared. Several authors have pointed out that it does not make sense to apply the integrated discrimination improvement and discrimination slope when working with mis-calibrated models, whereas others have raised concerns about the ability of improving discrimination slope without adding new information. In this paper, we show that under certain assumptions the discrimination slope is asymptotically related to two other R-squared measures, one of which is a rescaled version of the Brier score, known to be proper. Furthermore, we illustrate how a simple recalibration makes the slope equal to the rescaled Brier R-squared metric. We also show that the discrimination slope can be interpreted as a measure of reduction in expected regret for the Gini-Brier regret function. Using theoretical and practical examples, we illustrate how all of these metrics are affected by different levels of model mis-calibration. In particular, we demonstrate that simple recalibration ascertaining calibration in-the-large and calibration slope equal to 1 are not sufficient to correct for some forms of mis-calibration. We conclude that R-squared metrics, including the discrimination slope, offer an attractive choice for quantifying model performance as long as one accounts for their sensitivity to model calibration. Copyright © 2016 John Wiley & Sons, Ltd.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Stat Med

DOI

EISSN

1097-0258

Publication Date

December 10, 2017

Volume

36

Issue

28

Start / End Page

4482 / 4490

Location

England

Related Subject Headings

  • Statistics & Probability
  • Risk Assessment
  • Regression Analysis
  • Prognosis
  • Linear Models
  • Humans
  • Calibration
  • Bias
  • Area Under Curve
  • 4905 Statistics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Pencina, M. J., Fine, J. P., & D’Agostino, R. B. (2017). Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration. Stat Med, 36(28), 4482–4490. https://doi.org/10.1002/sim.7139
Pencina, Michael J., Jason P. Fine, and Ralph B. D’Agostino. “Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration.Stat Med 36, no. 28 (December 10, 2017): 4482–90. https://doi.org/10.1002/sim.7139.
Pencina MJ, Fine JP, D’Agostino RB. Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration. Stat Med. 2017 Dec 10;36(28):4482–90.
Pencina, Michael J., et al. “Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration.Stat Med, vol. 36, no. 28, Dec. 2017, pp. 4482–90. Pubmed, doi:10.1002/sim.7139.
Pencina MJ, Fine JP, D’Agostino RB. Discrimination slope and integrated discrimination improvement - properties, relationships and impact of calibration. Stat Med. 2017 Dec 10;36(28):4482–4490.
Journal cover image

Published In

Stat Med

DOI

EISSN

1097-0258

Publication Date

December 10, 2017

Volume

36

Issue

28

Start / End Page

4482 / 4490

Location

England

Related Subject Headings

  • Statistics & Probability
  • Risk Assessment
  • Regression Analysis
  • Prognosis
  • Linear Models
  • Humans
  • Calibration
  • Bias
  • Area Under Curve
  • 4905 Statistics