Skip to main content
Journal cover image

Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020.

Publication ,  Journal Article
Perez Alday, EA; Gu, A; J Shah, A; Robichaux, C; Ian Wong, A-K; Liu, C; Liu, F; Bahrami Rad, A; Elola, A; Seyedi, S; Li, Q; Sharma, A ...
Published in: Physiol Meas
January 1, 2021

OBJECTIVE: Vast 12-lead ECGs repositories provide opportunities to develop new machine learning approaches for creating accurate and automatic diagnostic systems for cardiac abnormalities. However, most 12-lead ECG classification studies are trained, tested, or developed in single, small, or relatively homogeneous datasets. In addition, most algorithms focus on identifying small numbers of cardiac arrhythmias that do not represent the complexity and difficulty of ECG interpretation. This work addresses these issues by providing a standard, multi-institutional database and a novel scoring metric through a public competition: the PhysioNet/Computing in Cardiology Challenge 2020. APPROACH: A total of 66 361 12-lead ECG recordings were sourced from six hospital systems from four countries across three continents; 43 101 recordings were posted publicly with a focus on 27 diagnoses. For the first time in a public competition, we required teams to publish open-source code for both training and testing their algorithms, ensuring full scientific reproducibility. MAIN RESULTS: A total of 217 teams submitted 1395 algorithms during the Challenge, representing a diversity of approaches for identifying cardiac abnormalities from both academia and industry. As with previous Challenges, high-performing algorithms exhibited significant drops ([Formula: see text]10%) in performance on the hidden test data. SIGNIFICANCE: Data from diverse institutions allowed us to assess algorithmic generalizability. A novel evaluation metric considered different misclassification errors for different cardiac abnormalities, capturing the outcomes and risks of different diagnoses. Requiring both trained models and code for training models improved the generalizability of submissions, setting a new bar in reproducibility for public data science competitions.

Duke Scholars

Published In

Physiol Meas

DOI

EISSN

1361-6579

Publication Date

January 1, 2021

Volume

41

Issue

12

Start / End Page

124003

Location

England

Related Subject Headings

  • Reproducibility of Results
  • Middle Aged
  • Male
  • Humans
  • Female
  • Electrocardiography
  • Databases, Factual
  • Cardiology
  • Biomedical Engineering
  • Arrhythmias, Cardiac
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Perez Alday, E. A., Gu, A., J Shah, A., Robichaux, C., Ian Wong, A.-K., Liu, C., … Reyna, M. A. (2021). Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas, 41(12), 124003. https://doi.org/10.1088/1361-6579/abc960
Perez Alday, Erick A., Annie Gu, Amit J Shah, Chad Robichaux, An-Kwok Ian Wong, Chengyu Liu, Feifei Liu, et al. “Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020.Physiol Meas 41, no. 12 (January 1, 2021): 124003. https://doi.org/10.1088/1361-6579/abc960.
Perez Alday EA, Gu A, J Shah A, Robichaux C, Ian Wong A-K, Liu C, et al. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas. 2021 Jan 1;41(12):124003.
Perez Alday, Erick A., et al. “Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020.Physiol Meas, vol. 41, no. 12, Jan. 2021, p. 124003. Pubmed, doi:10.1088/1361-6579/abc960.
Perez Alday EA, Gu A, J Shah A, Robichaux C, Ian Wong A-K, Liu C, Liu F, Bahrami Rad A, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD, Reyna MA. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas. 2021 Jan 1;41(12):124003.
Journal cover image

Published In

Physiol Meas

DOI

EISSN

1361-6579

Publication Date

January 1, 2021

Volume

41

Issue

12

Start / End Page

124003

Location

England

Related Subject Headings

  • Reproducibility of Results
  • Middle Aged
  • Male
  • Humans
  • Female
  • Electrocardiography
  • Databases, Factual
  • Cardiology
  • Biomedical Engineering
  • Arrhythmias, Cardiac