Skip to main content

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Publication ,  Journal Article
Daniel, E; Quinn, GE; Hildebrand, PL; Ells, A; Hubbard, GB; Capone, A; Martin, ER; Ostroff, CP; Smith, E; Pistilli, M; Ying, G-S ...
Published in: JAMA Ophthalmol
June 2015

IMPORTANCE: Measurable competence derived from comprehensive and advanced training in grading digital images is critical in studies using a reading center to evaluate retinal fundus images from infants at risk for retinopathy of prematurity (ROP). Details of certification for nonphysician trained readers (TRs) have not yet been described. OBJECTIVE: To describe a centralized system for grading ROP digital images by TRs in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study. DESIGN, SETTING, AND PARTICIPANTS: Multicenter observational cohort study conducted from July 1, 2010, to June 30, 2014. The TRs were trained by experienced ROP specialists and certified to detect ROP morphology in digital retinal images under supervision of an ophthalmologist reading center director. An ROP reading center was developed with standard hardware, secure Internet access, and customized image viewing software with an electronic grading form. A detailed protocol for grading was developed. Based on results of TR gradings, a computerized algorithm determined whether referral-warranted ROP (RW-ROP; defined as presence of plus disease, zone I ROP, and stage 3 or worse ROP) was present in digital images from infants with birth weight less than 1251 g enrolled from May 25, 2011, through October 31, 2013. Independent double grading was done by the TRs with adjudication of discrepant fields performed by the reading center director. EXPOSURE: Digital retinal images. MAIN OUTCOMES AND MEASURES: Intragrader and intergrader variability and monitoring for temporal drift. RESULTS: Four TRs underwent rigorous training and certification. A total of 5520 image sets were double graded, with 24.5% requiring adjudication for at least 1 component of RW-ROP. For individual RW-ROP components, the adjudication rate was 3.9% for plus disease, 12.4% for zone I ROP, and 16.9% for stage 3 or worse ROP. The weighted κ for intergrader agreement (n = 80 image sets) was 0.72 (95% CI, 0.52-0.93) for RW-ROP, 0.57 (95% CI, 0.37-0.77) for plus disease, 0.43 (95% CI, 0.24-0.63) for zone I ROP, and 0.67 (95% CI, 0.47-0.88) for stage 3 or worse ROP. The weighted κ for grade-regrade agreement was 0.77 (95% CI, 0.57-0.97) for RW-ROP, 0.87 (95% CI, 0.67-1.00) for plus disease, 0.70 (95% CI, 0.51-0.90) for zone I ROP, and 0.77 (95% CI, 0.57-0.97) for stage 3 or worse ROP. CONCLUSIONS AND RELEVANCE: These data suggest that the e-ROP system for training and certifying nonphysicians to grade ROP images under the supervision of a reading center director reliably detects potentially serious ROP with good intragrader and intergrader consistency and minimal temporal drift.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

JAMA Ophthalmol

DOI

EISSN

2168-6173

Publication Date

June 2015

Volume

133

Issue

6

Start / End Page

675 / 682

Location

United States

Related Subject Headings

  • Telemedicine
  • Retinopathy of Prematurity
  • Reproducibility of Results
  • Quality Assurance, Health Care
  • Ophthalmology & Optometry
  • Ophthalmology
  • Observer Variation
  • Infant, Very Low Birth Weight
  • Infant, Newborn
  • Humans
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Daniel, E., Quinn, G. E., Hildebrand, P. L., Ells, A., Hubbard, G. B., Capone, A., … e-ROP Cooperative Group, . (2015). Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study. JAMA Ophthalmol, 133(6), 675–682. https://doi.org/10.1001/jamaophthalmol.2015.0460
Daniel, Ebenezer, Graham E. Quinn, P Lloyd Hildebrand, Anna Ells, G Baker Hubbard, Antonio Capone, E Revell Martin, et al. “Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.JAMA Ophthalmol 133, no. 6 (June 2015): 675–82. https://doi.org/10.1001/jamaophthalmol.2015.0460.
Daniel E, Quinn GE, Hildebrand PL, Ells A, Hubbard GB, Capone A, et al. Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study. JAMA Ophthalmol. 2015 Jun;133(6):675–82.
Daniel, Ebenezer, et al. “Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.JAMA Ophthalmol, vol. 133, no. 6, June 2015, pp. 675–82. Pubmed, doi:10.1001/jamaophthalmol.2015.0460.
Daniel E, Quinn GE, Hildebrand PL, Ells A, Hubbard GB, Capone A, Martin ER, Ostroff CP, Smith E, Pistilli M, Ying G-S, e-ROP Cooperative Group. Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study. JAMA Ophthalmol. 2015 Jun;133(6):675–682.

Published In

JAMA Ophthalmol

DOI

EISSN

2168-6173

Publication Date

June 2015

Volume

133

Issue

6

Start / End Page

675 / 682

Location

United States

Related Subject Headings

  • Telemedicine
  • Retinopathy of Prematurity
  • Reproducibility of Results
  • Quality Assurance, Health Care
  • Ophthalmology & Optometry
  • Ophthalmology
  • Observer Variation
  • Infant, Very Low Birth Weight
  • Infant, Newborn
  • Humans