Skip to main content

Interpreting write performance of supercomputer I/O systems with regression models

Publication ,  Conference
Xie, B; Tan, Z; Carns, P; Chase, J; Harms, K; Lofstead, J; Oral, S; Vazhkudai, SS; Wang, F
Published in: Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021
May 1, 2021

This work seeks to advance the state of the art in HPC I/O performance analysis and interpretation. In particular, we demonstrate effective techniques to: (1) model output performance in the presence of I/O interference from production loads; (2) build features from write patterns and key parameters of the system architecture and configurations; (3) employ suitable machine learning algorithms to improve model accuracy. We train models with five popular regression algorithms and conduct experiments on two distinct production HPC platforms. We find that the lasso and random forest models predict output performance with high accuracy on both of the target systems. We also explore use of the models to guide adaptation in I/O middleware systems, and show potential for improvements of at least 15% from model-guided adaptation on 70% of samples, and improvements up to 10 × on some samples for both of the target systems.

Duke Scholars

Published In

Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021

DOI

ISBN

9781665440660

Publication Date

May 1, 2021

Start / End Page

557 / 566
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Xie, B., Tan, Z., Carns, P., Chase, J., Harms, K., Lofstead, J., … Wang, F. (2021). Interpreting write performance of supercomputer I/O systems with regression models. In Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021 (pp. 557–566). https://doi.org/10.1109/IPDPS49936.2021.00064
Xie, B., Z. Tan, P. Carns, J. Chase, K. Harms, J. Lofstead, S. Oral, S. S. Vazhkudai, and F. Wang. “Interpreting write performance of supercomputer I/O systems with regression models.” In Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021, 557–66, 2021. https://doi.org/10.1109/IPDPS49936.2021.00064.
Xie B, Tan Z, Carns P, Chase J, Harms K, Lofstead J, et al. Interpreting write performance of supercomputer I/O systems with regression models. In: Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021. 2021. p. 557–66.
Xie, B., et al. “Interpreting write performance of supercomputer I/O systems with regression models.” Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021, 2021, pp. 557–66. Scopus, doi:10.1109/IPDPS49936.2021.00064.
Xie B, Tan Z, Carns P, Chase J, Harms K, Lofstead J, Oral S, Vazhkudai SS, Wang F. Interpreting write performance of supercomputer I/O systems with regression models. Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021. 2021. p. 557–566.

Published In

Proceedings - 2021 IEEE 35th International Parallel and Distributed Processing Symposium, IPDPS 2021

DOI

ISBN

9781665440660

Publication Date

May 1, 2021

Start / End Page

557 / 566