Evaluation criteria for human-automation performance metrics
Book Section
Previous research has identified broad metric classes for human-automation performance in order to facilitate metric selection, as well as understanding and comparing research results. However, there is still a lack of a systematic method for selecting the most efficient set of metrics when designing evaluation experiments. This chapter identifies and presents a list of evaluation criteria that can help determine the quality of a metric in terms of experimental constraints, comprehensive understanding, construct validity, statistical efficiency, and measurement technique efficiency. Based on the evaluation criteria, a comprehensive list of potential metric costs and benefits is generated. The evaluation criteria, along with the list of metric costs and benefits, and the existing generic metric classes provide a foundation for the development of a cost-benefit analysis approach that can be used for metric selection. © 2009 Springer-Verlag US.
Full Text
Duke Authors
Cited Authors
- Donmez, B; Pina, PE; Cummings, ML
Published Date
- January 1, 2009
Start / End Page
- 21 - 40
International Standard Book Number 13 (ISBN-13)
- 9781441904911
Digital Object Identifier (DOI)
- 10.1007/978-1-4419-0492-8_2
Citation Source
- Scopus