Predicting output performance of a petascale supercomputer


Conference Paper

© 2017 Association for Computing Machinery. In this paper, we develop a predictive model useful for output performance prediction of supercomputer file systems under production load. Our target environment is Titan-the 3rd fastest supercomputer in the world-and its Lustre-based multi-stage write path. We observe from Titan that although output performance is highly variable at small time scales, the mean performance is stable and consistent over typical application run times. Moreover, we find that output performance is non-linearly related to its correlated parameters due to interference and saturation on individual stages on the path. These observations enable us to build a predictive model of expected write times of output patterns and I/O configurations, using feature transformations to capture non-linear relationships. We identify the candidate features based on the structure of the Lustre/Titan write path, and use feature transformation functions to produce a model space with 135,000 candidate models. By searching for the minimal mean square error in this space we identify a good model and show that it is effective.

Full Text

Duke Authors

Cited Authors

  • Xie, B; Huang, Y; Chase, JS; Choi, JY; Klasky, S; Lofstead, J; Oral, S

Published Date

  • June 26, 2017

Published In

  • Hpdc 2017 Proceedings of the 26th International Symposium on High Performance Parallel and Distributed Computing

Start / End Page

  • 181 - 192

International Standard Book Number 13 (ISBN-13)

  • 9781450346993

Digital Object Identifier (DOI)

  • 10.1145/3078597.3078614

Citation Source

  • Scopus