Quantile evaluation, sensitivity to bracketing, and sharing business payoffs

Journal Article (Journal Article)

From forecasting competitions to conditional value-at-risk requirements, the use of multiple quantile assessments is growing in practice. To evaluate them, we use a rule from the general class of proper scoring rules for a forecaster's multiple quantiles of a single uncertain quantity of interest. The general rule is additive in the component scores. Each component contains a function that measures its quantile's distance from the realization and weights its contribution to the overall score. To determine this function, we propose that the score of a group's combined quantile should be better than that of a randomly selected forecaster's quantile only when the forecasters bracket the realization (i.e., their quantiles do not fall on the same side of the realization). If a score satisfies this property, we say it is sensitive to bracketing. We characterize the class of proper scoring rules that is sensitive to bracketing when the decision maker uses a generalized average to combine forecasters' quantiles. Finally, we show how weights can be set to match the payoffs in many important business contexts.

Full Text

Duke Authors

Cited Authors

  • Grushka-Cockayne, Y; Lichtendahl, KC; Jose, VRR; Winkler, RL

Published Date

  • May 1, 2017

Published In

Volume / Issue

  • 65 / 3

Start / End Page

  • 712 - 728

Electronic International Standard Serial Number (EISSN)

  • 1526-5463

International Standard Serial Number (ISSN)

  • 0030-364X

Digital Object Identifier (DOI)

  • 10.1287/opre.2017.1588

Citation Source

  • Scopus