Evaluation criteria for human-automation performance metrics
Previous research has identified broad metric classes for humanautomation performance in order to facilitate metric selection, as well as understanding and comparing research results. However, there is still a lack of a systematic method for selecting the most efficient set of metrics when designing evaluation experiments. This chapter identifies and presents a list of evaluation criteria that can help determine the quality of a metric in terms of experimental constraints, comprehensive understanding, construct validity, statistical efficiency, and measurement technique efficiency. Based on the evaluation criteria, a comprehensive list of potential metric costs and benefits is generated. The evaluation criteria, along with the list of metric costs and benefits, and the existing generic metric classes provide a foundation for the development of a cost-benefit analysis approach that can be used for metric selection. © 2009 Springer Science+Business Media, LLC.
Donmez, B; Pina, PE; Cummings, ML
- Performance Evaluation and Benchmarking of Intelligent Systems
Start / End Page
International Standard Book Number 13 (ISBN-13)
Digital Object Identifier (DOI)