Reliable evaluation of performance level for computer-aided diagnostic scheme.


Journal Article

RATIONALE AND OBJECTIVES: Computer-aided diagnostic (CAD) schemes have been developed for assisting radiologists in the detection of various lesions in medical images. The reliable evaluation of CAD schemes is an important task in the field of CAD research. MATERIALS AND METHODS: Many evaluation approaches have been proposed for evaluating the performance of various CAD schemes in the past. However, some important issues in the evaluation of CAD schemes have not been systematically analyzed. The first important issue is the analysis and comparison of various evaluation methods in terms of certain characteristics. The second includes the analysis of pitfalls in the incorrect use of various evaluation methods and the effective approaches to the reduction of the bias and variance caused by these pitfalls. We attempt to address the first important issue in details in this article by conducting Monte Carlo simulation experiments, and to discuss the second issue in the Discussion section. RESULTS: No single evaluation method is universally superior to the others; different situations of CAD applications require different evaluation methods, as recommended in this article. Bias and variance in the estimated performance levels caused by various pitfalls can be reduced considerably by the correct use of good evaluation methods. CONCLUSIONS: This article would be useful to researchers in the field of CAD research for selecting appropriate evaluation methods and for improving the reliability of the estimated performance of their CAD schemes.

Full Text

Duke Authors

Cited Authors

  • Li, Q

Published Date

  • August 2007

Published In

Volume / Issue

  • 14 / 8

Start / End Page

  • 985 - 991

PubMed ID

  • 17659245

Pubmed Central ID

  • 17659245

International Standard Serial Number (ISSN)

  • 1076-6332

Digital Object Identifier (DOI)

  • 10.1016/j.acra.2007.04.015


  • eng

Conference Location

  • United States