Skip to main content

Efficient statistical validation of machine learning systems for autonomous driving

Publication ,  Conference
Shi, W; Alawieh, MB; Li, X; Yu, H; Arechiga, N; Tomatsu, N
Published in: IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD
November 7, 2016

Today's automotive industry is making a bold move to equip vehicles with intelligent driver assistance features. A modern automobile is now equipped with a powerful computing platform to run multiple machine learning algorithms for environment perception (e.g., pedestrian detection) and motion control (e.g., vehicle stabilization). These machine learning systems must be highly robust with extremely small failure rate in order to ensure safe and reliable driving. In this paper, we propose a novel Subset Sampling (SUS) algorithm to efficiently validate a machine learning system. In particular, a Markov Chain Monte Carlo algorithm based on graph mapping is developed to accurately estimate the rare failure rate with a minimal amount of test data, thereby minimizing the validation cost. Our numerical experiments show that SUS achieves 15.2x runtime speed-up over the conventional brute-force Monte Carlo method.

Duke Scholars

Published In

IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD

DOI

ISSN

1092-3152

Publication Date

November 7, 2016

Volume

07-10-November-2016
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Shi, W., Alawieh, M. B., Li, X., Yu, H., Arechiga, N., & Tomatsu, N. (2016). Efficient statistical validation of machine learning systems for autonomous driving. In IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD (Vol. 07-10-November-2016). https://doi.org/10.1145/2966986.2980077
Shi, W., M. B. Alawieh, X. Li, H. Yu, N. Arechiga, and N. Tomatsu. “Efficient statistical validation of machine learning systems for autonomous driving.” In IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD, Vol. 07-10-November-2016, 2016. https://doi.org/10.1145/2966986.2980077.
Shi W, Alawieh MB, Li X, Yu H, Arechiga N, Tomatsu N. Efficient statistical validation of machine learning systems for autonomous driving. In: IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD. 2016.
Shi, W., et al. “Efficient statistical validation of machine learning systems for autonomous driving.” IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD, vol. 07-10-November-2016, 2016. Scopus, doi:10.1145/2966986.2980077.
Shi W, Alawieh MB, Li X, Yu H, Arechiga N, Tomatsu N. Efficient statistical validation of machine learning systems for autonomous driving. IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD. 2016.

Published In

IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD

DOI

ISSN

1092-3152

Publication Date

November 7, 2016

Volume

07-10-November-2016