The Separation Plot: A New Visual Method for Evaluating the Fit of Binary Models

Published

Journal Article

We present a visual method for assessing the predictive power of models with binary outcomes. This technique allows the analyst to evaluate model fit based upon the models' ability to consistently match high-probability predictions to actual occurrences of the event of interest, and low-probability predictions to nonoccurrences of the event of interest. Unlike existing methods for assessing predictive power for logit and probit models such as Percent Correctly Predicted statistics, Brier scores, and the ROC plot, our "separation plot" has the advantage of producing a visual display that is informative and easy to explain to a general audience, while also remaining insensitive to the often arbitrary probability thresholds that are used to distinguish between predicted events and nonevents. We demonstrate the effectiveness of this technique in building predictive models in a number of different areas of political research. © 2011, Midwest Political Science Association.

Full Text

Duke Authors

Cited Authors

  • Greenhill, B; Ward, MD; Sacks, A

Published Date

  • October 1, 2011

Published In

Volume / Issue

  • 55 / 4

Start / End Page

  • 991 - 1002

Electronic International Standard Serial Number (EISSN)

  • 1540-5907

International Standard Serial Number (ISSN)

  • 0092-5853

Digital Object Identifier (DOI)

  • 10.1111/j.1540-5907.2011.00525.x

Citation Source

  • Scopus