Multi-program benchmark definition
Conference Paper
Although definition of single-program benchmarks is relatively straight-forward-a benchmark is a program plus a specific input-definition of multi-program benchmarks is more complex. Each program may have a different runtime and they may have different interactions depending on how they align with each other. While prior work has focused on sampling multiprogram benchmarks, little attention has been paid to defining the benchmarks in their entirety. In this work, we propose a four-tuple that formally defines multi-program benchmarks in a well-defined way. We then examine how four different classes of benchmarks created by varying the elements of this tuple align with real-world use-cases. We evaluate the impact of these variations on real hardware, and see drastic variations in results between different benchmarks constructed from the same programs. Notable differences include significant speedups versus slowdowns (e.g., +57% vs -5% or +26% vs -18%), and large differences in magnitude even when the results are in the same direction (e.g., 67% versus 11%).
Full Text
Duke Authors
Cited Authors
- Jacobvitz, AN; Hilton, AD; Sorin, DJ
Published Date
- April 27, 2015
Published In
- Ispass 2015 Ieee International Symposium on Performance Analysis of Systems and Software
Start / End Page
- 72 - 82
International Standard Book Number 13 (ISBN-13)
- 9781479919567
Digital Object Identifier (DOI)
- 10.1109/ISPASS.2015.7095786
Citation Source
- Scopus