Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography.

Journal Article (Journal Article)

Recently, a class of no-gold-standard (NGS) techniques have been proposed to evaluate quantitative imaging methods using patient data. These techniques provide figures of merit (FoMs) quantifying the precision of the estimated quantitative value without requiring repeated measurements and without requiring a gold standard. However, applying these techniques to patient data presents several practical difficulties including assessing the underlying assumptions, accounting for patient-sampling-related uncertainty, and assessing the reliability of the estimated FoMs. To address these issues, we propose statistical tests that provide confidence in the underlying assumptions and in the reliability of the estimated FoMs. Furthermore, the NGS technique is integrated within a bootstrap-based methodology to account for patient-sampling-related uncertainty. The developed NGS framework was applied to evaluate four methods for segmenting lesions from F-Fluoro-2-deoxyglucose positron emission tomography images of patients with head-and-neck cancer on the task of precisely measuring the metabolic tumor volume. The NGS technique consistently predicted the same segmentation method as the most precise method. The proposed framework provided confidence in these results, even when gold-standard data were not available. The bootstrap-based methodology indicated improved performance of the NGS technique with larger numbers of patient studies, as was expected, and yielded consistent results as long as data from more than 80 lesions were available for the analysis.

Full Text

Duke Authors

Cited Authors

  • Jha, AK; Mena, E; Caffo, B; Ashrafinia, S; Rahmim, A; Frey, E; Subramaniam, RM

Published Date

  • January 2017

Published In

Volume / Issue

  • 4 / 1

Start / End Page

  • 011011 -

PubMed ID

  • 28331883

Pubmed Central ID

  • PMC5335899

International Standard Serial Number (ISSN)

  • 2329-4302

Digital Object Identifier (DOI)

  • 10.1117/1.JMI.4.1.011011


  • eng

Conference Location

  • United States