IQCar: Inter-query contention analyzer for data analytics frameworks

Published

Conference Paper

© 2019 Association for Computing Machinery. Resource interferences caused by concurrent queries is one of the key reasons for unpredictable performance and missed workload SLAs in cluster computing systems. Analyzing these inter-query resource interactions is critical in order to answer time-sensitive questions like 'who is creating resource conflicts to my query'. More importantly, diagnosing whether the resource blocked times of a 'victim' query are caused by other queries or some other external factor can help the database administrator narrow down the many possibilities of query performance degradation. We introduce iQCAR, an inter-Query Contention Analyzer, that attributes blame for the slowdown of a query to concurrent queries. iQCAR models the resource conflicts using a multi-level directed acyclic graph that can help administrators compare impacts from concurrent queries, identify most contentious queries, resources and hosts in an online execution for a selected time window. Our experiments using TPCDS queries on Apache Spark show that our approach is substantially more accurate than other methods based on overlap time between concurrent queries.

Full Text

Duke Authors

Cited Authors

  • Kalmegh, P; Babu, S; Roy, S

Published Date

  • June 25, 2019

Published In

Start / End Page

  • 918 - 935

International Standard Serial Number (ISSN)

  • 0730-8078

International Standard Book Number 13 (ISBN-13)

  • 9781450356435

Digital Object Identifier (DOI)

  • 10.1145/3299869.3319904

Citation Source

  • Scopus