Skip to main content

On incomplete learning and certainty-equivalence control

Publication ,  Journal Article
Bora Keskin, N; Zeevi, A
Published in: Operations Research
July 1, 2018

We consider a dynamic learning problem where a decision maker sequentially selects a control and observes a response variable that depends on chosen control and an unknown sensitivity parameter. After every observation, the decision maker updates his or her estimate of the unknown parameter and uses a certainty-equivalence decision rule to determine subsequent controls based on this estimate. We show that under this certainty-equivalence learning policy the parameter estimates converge with positive probability to an uninformative fixed point that can differ from the true value of the unknown parameter; a phenomenon that will be referred to as incomplete learning. In stark contrast, it will be shown that this certainty-equivalence policy may avoid incomplete learning if the parameter value of interest "drifts away" from the uninformative fixed point at a critical rate. Finally, we prove that one can adaptively limit the learning memory to improve the accuracy of the certainty-equivalence policy in both static (estimation), as well as slowly varying (tracking) environments, without relying on forced exploration.

Duke Scholars

Published In

Operations Research

DOI

EISSN

1526-5463

ISSN

0030-364X

Publication Date

July 1, 2018

Volume

66

Issue

4

Start / End Page

1136 / 1167

Related Subject Headings

  • Operations Research
  • 3507 Strategy, management and organisational behaviour
  • 1503 Business and Management
  • 0802 Computation Theory and Mathematics
  • 0102 Applied Mathematics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Bora Keskin, N., & Zeevi, A. (2018). On incomplete learning and certainty-equivalence control. Operations Research, 66(4), 1136–1167. https://doi.org/10.1287/opre.2017.1713
Bora Keskin, N., and A. Zeevi. “On incomplete learning and certainty-equivalence control.” Operations Research 66, no. 4 (July 1, 2018): 1136–67. https://doi.org/10.1287/opre.2017.1713.
Bora Keskin N, Zeevi A. On incomplete learning and certainty-equivalence control. Operations Research. 2018 Jul 1;66(4):1136–67.
Bora Keskin, N., and A. Zeevi. “On incomplete learning and certainty-equivalence control.” Operations Research, vol. 66, no. 4, July 2018, pp. 1136–67. Scopus, doi:10.1287/opre.2017.1713.
Bora Keskin N, Zeevi A. On incomplete learning and certainty-equivalence control. Operations Research. 2018 Jul 1;66(4):1136–1167.

Published In

Operations Research

DOI

EISSN

1526-5463

ISSN

0030-364X

Publication Date

July 1, 2018

Volume

66

Issue

4

Start / End Page

1136 / 1167

Related Subject Headings

  • Operations Research
  • 3507 Strategy, management and organisational behaviour
  • 1503 Business and Management
  • 0802 Computation Theory and Mathematics
  • 0102 Applied Mathematics