Evaluation criteria for human-automation performance metrics

Book Section

Previous research has identified broad metric classes for human-automation performance in order to facilitate metric selection, as well as understanding and comparing research results. However, there is still a lack of a systematic method for selecting the most efficient set of metrics when designing evaluation experiments. This chapter identifies and presents a list of evaluation criteria that can help determine the quality of a metric in terms of experimental constraints, comprehensive understanding, construct validity, statistical efficiency, and measurement technique efficiency. Based on the evaluation criteria, a comprehensive list of potential metric costs and benefits is generated. The evaluation criteria, along with the list of metric costs and benefits, and the existing generic metric classes provide a foundation for the development of a cost-benefit analysis approach that can be used for metric selection. © 2009 Springer-Verlag US.

Full Text

Duke Authors

Cited Authors

  • Donmez, B; Pina, PE; Cummings, ML

Published Date

  • January 1, 2009

Start / End Page

  • 21 - 40

International Standard Book Number 13 (ISBN-13)

  • 9781441904911

Digital Object Identifier (DOI)

  • 10.1007/978-1-4419-0492-8_2

Citation Source

  • Scopus