Skip to main content

Benchmark Set Reduction for Cheap Empirical Algorithmic Studies

Publication ,  Conference
Mısır, M
Published in: 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings
January 1, 2021

The present paper introduces a benchmark set reduction strategy that can degrade the experimental evaluation cost for the algorithmic studies. Algorithm design is an iterative development process within in a test-revise loop. Starting a devised algorithm's initial version, it needs to be tested for revealing its advantages and drawbacks. Its shortcomings lead the designers to modify the algorithm. Then, the modified algorithm is again tested. This two-step pipeline is repeated until the algorithm meets the expectations such as delivering the state-of-the-art results on a set of benchmarks for a specific problem. That recurring testing step can be a burden for whom with limited computational resources. The mentioned computational cost can mainly occur due either high per instance runtime cost or having a large benchmark set. This study focuses on the cases when a target algorithm needs to be assessed across large benchmark sets. The idea is to automatically extract problem instance representation through an instance-algorithm performance data. The derived representation in the form of latent features is utilized to determine a small yet a representative subset of a given large instance set. The proposed strategy is investigated on the Traveling Thief Problem of 9720 instances. The corresponding performance data is collected by the help of 21 TTP algorithms. The resulting computational analysis showed that the proposed method is capable of substantially minimizing the benchmark instance set size.

Duke Scholars

Published In

2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings

DOI

ISBN

9781728183923

Publication Date

January 1, 2021

Start / End Page

871 / 877
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Mısır, M. (2021). Benchmark Set Reduction for Cheap Empirical Algorithmic Studies. In 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings (pp. 871–877). https://doi.org/10.1109/CEC45853.2021.9505012
Mısır, M. “Benchmark Set Reduction for Cheap Empirical Algorithmic Studies.” In 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings, 871–77, 2021. https://doi.org/10.1109/CEC45853.2021.9505012.
Mısır M. Benchmark Set Reduction for Cheap Empirical Algorithmic Studies. In: 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings. 2021. p. 871–7.
Mısır, M. “Benchmark Set Reduction for Cheap Empirical Algorithmic Studies.” 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings, 2021, pp. 871–77. Scopus, doi:10.1109/CEC45853.2021.9505012.
Mısır M. Benchmark Set Reduction for Cheap Empirical Algorithmic Studies. 2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings. 2021. p. 871–877.

Published In

2021 IEEE Congress on Evolutionary Computation, CEC 2021 - Proceedings

DOI

ISBN

9781728183923

Publication Date

January 1, 2021

Start / End Page

871 / 877