Evaluating quantile assessments

Published

Journal Article

Quantile assessments are commonly encountered in the elicitation of probability distributions in decision analysis, forecasting, and risk analysis. Scoring rules have been developed to provide ex ante incentives for careful and truthful assessments and ex post evaluation measures in the context of probability assessment. We show that these scoring rules designed for probability assessment provide inappropriate incentives if used for quantile assessment. We investigate the properties of a linear family of scoring rules that are intended specifically for quantile assessment (including the assessment of multiple quantiles) and can be related to a realistic decision-making problem. These rules provide proper incentives for quantile assessment and yield higher expected scores for distributions that are more informative in the sense of having less dispersion. We discuss the special case of interval forecasts and a generalization involving transformations, and we briefly mention other possible extensions. © 2009 INFORMS.

Full Text

Duke Authors

Cited Authors

  • Jose, VRR; Winkler, RL

Published Date

  • September 1, 2009

Published In

Volume / Issue

  • 57 / 5

Start / End Page

  • 1287 - 1297

Electronic International Standard Serial Number (EISSN)

  • 1526-5463

International Standard Serial Number (ISSN)

  • 0030-364X

Digital Object Identifier (DOI)

  • 10.1287/opre.1080.0665

Citation Source

  • Scopus