Bias and variance approximation in value function estimates


Journal Article

We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance, which can then be used to derive confidence intervals around the value function estimates. We illustrate and validate our findings using a large database describing the transaction and mailing histories for customers of a mail-order catalog firm. © 2007 INFORMS.

Full Text

Duke Authors

Cited Authors

  • Mannor, S; Simester, D; Sun, P; Tsitsiklis, JN

Published Date

  • February 1, 2007

Published In

Volume / Issue

  • 53 / 2

Start / End Page

  • 308 - 322

Electronic International Standard Serial Number (EISSN)

  • 1526-5501

International Standard Serial Number (ISSN)

  • 0025-1909

Digital Object Identifier (DOI)

  • 10.1287/mnsc.1060.0614

Citation Source

  • Scopus