Reinforcement Learning Methods in Public Health.

Journal Article (Review;Journal Article)

Purpose

Reinforcement learning (RL) is the subfield of machine learning focused on optimal sequential decision making under uncertainty. An optimal RL strategy maximizes cumulative utility by experimenting only if and when the information generated by experimentation is likely to outweigh associated short-term costs. RL represents a holistic approach to decision making that evaluates the impact of every action (ie, data collection, allocation of resources, and treatment assignment) in terms of short-term and long-term utility to stakeholders. Thus, RL is an ideal model for a number of complex decision problems that arise in public health, including resource allocation in a pandemic, monitoring or testing, and adaptive sampling for hidden populations. Nevertheless, although RL has been applied successfully in a wide range of domains, including precision medicine, it has not been widely adopted in public health. The purposes of this review are to introduce key ideas in RL and to identify challenges and opportunities associated with the application of RL in public health.

Methods

We provide a nontechnical review of the theoretical and methodologic underpinnings of RL. A running example of RL for the management of an infectious disease is used to illustrate ideas.

Findings

RL has the potential to make a transformative impact in a range of sequential decision problems in public health. By allocating resources if, when, and where they are most impactful, RL can improve health outcomes while reducing resource consumption.

Implications

Public health researchers and stakeholders should consider RL as a means of efficiently using data to inform optimal evidence-based decision making.

Full Text

Duke Authors

Cited Authors

  • Weltz, J; Volfovsky, A; Laber, EB

Published Date

  • January 2022

Published In

Volume / Issue

  • 44 / 1

Start / End Page

  • 139 - 154

PubMed ID

  • 35058056

Electronic International Standard Serial Number (EISSN)

  • 1879-114X

International Standard Serial Number (ISSN)

  • 0149-2918

Digital Object Identifier (DOI)

  • 10.1016/j.clinthera.2021.11.002

Language

  • eng