Bias and variance in value function estimation
Publication
, Journal Article
Mannor, S; Simester, D; Sun, P; Tsitsiklis, JN
Published in: Proceedings Twenty First International Conference on Machine Learning Icml 2004
December 1, 2004
We consider the bias and variance of value function estimation that are caused by using an empirical model instead of the true model. We analyze these bias and variance for Markov processes from a classical (frequentist) statistical point of view, and in a Bayesian setting. Using a second order approximation, we provide explicit expressions for the bias and variance in terms of the transition counts and the reward statistics. We present supporting experiments with artificial Markov chains and with a large transactional database provided by a mail-order catalog firm.
Duke Scholars
Published In
Proceedings Twenty First International Conference on Machine Learning Icml 2004
Publication Date
December 1, 2004
Start / End Page
568 / 575
Citation
APA
Chicago
ICMJE
MLA
NLM
Mannor, S., Simester, D., Sun, P., & Tsitsiklis, J. N. (2004). Bias and variance in value function estimation. Proceedings Twenty First International Conference on Machine Learning Icml 2004, 568–575.
Mannor, S., D. Simester, P. Sun, and J. N. Tsitsiklis. “Bias and variance in value function estimation.” Proceedings Twenty First International Conference on Machine Learning Icml 2004, December 1, 2004, 568–75.
Mannor S, Simester D, Sun P, Tsitsiklis JN. Bias and variance in value function estimation. Proceedings Twenty First International Conference on Machine Learning Icml 2004. 2004 Dec 1;568–75.
Mannor, S., et al. “Bias and variance in value function estimation.” Proceedings Twenty First International Conference on Machine Learning Icml 2004, Dec. 2004, pp. 568–75.
Mannor S, Simester D, Sun P, Tsitsiklis JN. Bias and variance in value function estimation. Proceedings Twenty First International Conference on Machine Learning Icml 2004. 2004 Dec 1;568–575.
Published In
Proceedings Twenty First International Conference on Machine Learning Icml 2004
Publication Date
December 1, 2004
Start / End Page
568 / 575