Skip to main content
Journal cover image

Machine learning for early detection of sepsis: an internal and temporal validation study.

Publication ,  Journal Article
Bedoya, AD; Futoma, J; Clement, ME; Corey, K; Brajer, N; Lin, A; Simons, MG; Gao, M; Nichols, M; Balu, S; Heller, K; Sendak, M; O'Brien, C
Published in: JAMIA Open
July 2020

OBJECTIVE: Determine if deep learning detects sepsis earlier and more accurately than other models. To evaluate model performance using implementation-oriented metrics that simulate clinical practice. MATERIALS AND METHODS: We trained internally and temporally validated a deep learning model (multi-output Gaussian process and recurrent neural network [MGP-RNN]) to detect sepsis using encounters from adult hospitalized patients at a large tertiary academic center. Sepsis was defined as the presence of 2 or more systemic inflammatory response syndrome (SIRS) criteria, a blood culture order, and at least one element of end-organ failure. The training dataset included demographics, comorbidities, vital signs, medication administrations, and labs from October 1, 2014 to December 1, 2015, while the temporal validation dataset was from March 1, 2018 to August 31, 2018. Comparisons were made to 3 machine learning methods, random forest (RF), Cox regression (CR), and penalized logistic regression (PLR), and 3 clinical scores used to detect sepsis, SIRS, quick Sequential Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS). Traditional discrimination statistics such as the C-statistic as well as metrics aligned with operational implementation were assessed. RESULTS: The training set and internal validation included 42 979 encounters, while the temporal validation set included 39 786 encounters. The C-statistic for predicting sepsis within 4 h of onset was 0.88 for the MGP-RNN compared to 0.836 for RF, 0.849 for CR, 0.822 for PLR, 0.756 for SIRS, 0.619 for NEWS, and 0.481 for qSOFA. MGP-RNN detected sepsis a median of 5 h in advance. Temporal validation assessment continued to show the MGP-RNN outperform all 7 clinical risk score and machine learning comparisons. CONCLUSIONS: We developed and validated a novel deep learning model to detect sepsis. Using our data elements and feature set, our modeling approach outperformed other machine learning methods and clinical scores.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

JAMIA Open

DOI

EISSN

2574-2531

Publication Date

July 2020

Volume

3

Issue

2

Start / End Page

252 / 260

Location

United States

Related Subject Headings

  • 4203 Health services and systems
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Bedoya, A. D., Futoma, J., Clement, M. E., Corey, K., Brajer, N., Lin, A., … O’Brien, C. (2020). Machine learning for early detection of sepsis: an internal and temporal validation study. JAMIA Open, 3(2), 252–260. https://doi.org/10.1093/jamiaopen/ooaa006
Bedoya, Armando D., Joseph Futoma, Meredith E. Clement, Kristin Corey, Nathan Brajer, Anthony Lin, Morgan G. Simons, et al. “Machine learning for early detection of sepsis: an internal and temporal validation study.JAMIA Open 3, no. 2 (July 2020): 252–60. https://doi.org/10.1093/jamiaopen/ooaa006.
Bedoya AD, Futoma J, Clement ME, Corey K, Brajer N, Lin A, et al. Machine learning for early detection of sepsis: an internal and temporal validation study. JAMIA Open. 2020 Jul;3(2):252–60.
Bedoya, Armando D., et al. “Machine learning for early detection of sepsis: an internal and temporal validation study.JAMIA Open, vol. 3, no. 2, July 2020, pp. 252–60. Pubmed, doi:10.1093/jamiaopen/ooaa006.
Bedoya AD, Futoma J, Clement ME, Corey K, Brajer N, Lin A, Simons MG, Gao M, Nichols M, Balu S, Heller K, Sendak M, O’Brien C. Machine learning for early detection of sepsis: an internal and temporal validation study. JAMIA Open. 2020 Jul;3(2):252–260.
Journal cover image

Published In

JAMIA Open

DOI

EISSN

2574-2531

Publication Date

July 2020

Volume

3

Issue

2

Start / End Page

252 / 260

Location

United States

Related Subject Headings

  • 4203 Health services and systems