Efficient statistical validation of machine learning systems for autonomous driving

Conference Paper

Today's automotive industry is making a bold move to equip vehicles with intelligent driver assistance features. A modern automobile is now equipped with a powerful computing platform to run multiple machine learning algorithms for environment perception (e.g., pedestrian detection) and motion control (e.g., vehicle stabilization). These machine learning systems must be highly robust with extremely small failure rate in order to ensure safe and reliable driving. In this paper, we propose a novel Subset Sampling (SUS) algorithm to efficiently validate a machine learning system. In particular, a Markov Chain Monte Carlo algorithm based on graph mapping is developed to accurately estimate the rare failure rate with a minimal amount of test data, thereby minimizing the validation cost. Our numerical experiments show that SUS achieves 15.2x runtime speed-up over the conventional brute-force Monte Carlo method.

Full Text

Duke Authors

Cited Authors

  • Shi, W; Alawieh, MB; Li, X; Yu, H; Arechiga, N; Tomatsu, N

Published Date

  • November 7, 2016

Published In

Volume / Issue

  • 07-10-November-2016 /

International Standard Serial Number (ISSN)

  • 1092-3152

International Standard Book Number 13 (ISBN-13)

  • 9781450344661

Digital Object Identifier (DOI)

  • 10.1145/2966986.2980077

Citation Source

  • Scopus