Skip to main content

Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy

Publication ,  Journal Article
Mazurowski, MA; Tourassi, GD
Published in: Proceedings of the International Joint Conference on Neural Networks
November 18, 2009

In this study, we investigated the relation between two popular classifier performance measures: area under the receiver operator characteristic curve and overall accuracy. We also evaluated the impact of class imbalance and number of examples in test set on this relation. We perform a set of experiments in which we train multiple neural networks and test them in various, well controlled conditions. The experimental results show that given a large and balanced test set, increase in one performance measure is a very good indicator of increase in the other measure. Furthermore increasing the total number of examples, while keeping the positive class prevalence constant generally increases the correlation between the two measures. Our results also indicate that increasing the extent of class imbalance in the test set has a detrimental effect on this correlation. ©2009 IEEE.

Duke Scholars

Published In

Proceedings of the International Joint Conference on Neural Networks

DOI

Publication Date

November 18, 2009

Start / End Page

2045 / 2049
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Mazurowski, M. A., & Tourassi, G. D. (2009). Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy. Proceedings of the International Joint Conference on Neural Networks, 2045–2049. https://doi.org/10.1109/IJCNN.2009.5178752
Mazurowski, M. A., and G. D. Tourassi. “Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy.” Proceedings of the International Joint Conference on Neural Networks, November 18, 2009, 2045–49. https://doi.org/10.1109/IJCNN.2009.5178752.
Mazurowski MA, Tourassi GD. Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy. Proceedings of the International Joint Conference on Neural Networks. 2009 Nov 18;2045–9.
Mazurowski, M. A., and G. D. Tourassi. “Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy.” Proceedings of the International Joint Conference on Neural Networks, Nov. 2009, pp. 2045–49. Scopus, doi:10.1109/IJCNN.2009.5178752.
Mazurowski MA, Tourassi GD. Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy. Proceedings of the International Joint Conference on Neural Networks. 2009 Nov 18;2045–2049.

Published In

Proceedings of the International Joint Conference on Neural Networks

DOI

Publication Date

November 18, 2009

Start / End Page

2045 / 2049