Skip to main content

Comprehensible counterfactual explanation on Kolmogorov-Smirnov test

Publication ,  Conference
Cong, Z; Chu, L; Yang, Y; Pei, J
Published in: Proceedings of the VLDB Endowment
January 1, 2021

The Kolmogorov-Smirnov (KS) test is popularly used in many applications, such as anomaly detection, astronomy, database security and AI systems. One challenge remained untouched is how we can obtain an explanation on why a test set fails the KS test. In this paper, we tackle the problem of producing counterfactual explanations for test data failing the KS test. Concept-wise, we propose the notion of most comprehensible counterfactual explanations, which accommodates both the KS test data and the user domain knowledge in producing explanations. Computation-wise, we develop an efficient algorithm MOCHE (for MOst CompreHensible Explanation) that avoids enumerating and checking an exponential number of subsets of the test set failing the KS test. MOCHE not only guarantees to produce the most comprehensible counterfactual explanations, but also is orders of magnitudes faster than the baselines. Experiment-wise, we present a systematic empirical study on a series of benchmark real datasets to verify the effectiveness, efficiency and scalability of most comprehensible counterfactual explanations and MOCHE.

Duke Scholars

Published In

Proceedings of the VLDB Endowment

DOI

EISSN

2150-8097

Publication Date

January 1, 2021

Volume

14

Issue

9

Start / End Page

1583 / 1596

Related Subject Headings

  • 4605 Data management and data science
  • 0807 Library and Information Studies
  • 0806 Information Systems
  • 0802 Computation Theory and Mathematics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Cong, Z., Chu, L., Yang, Y., & Pei, J. (2021). Comprehensible counterfactual explanation on Kolmogorov-Smirnov test. In Proceedings of the VLDB Endowment (Vol. 14, pp. 1583–1596). https://doi.org/10.14778/3461535.3461546
Cong, Z., L. Chu, Y. Yang, and J. Pei. “Comprehensible counterfactual explanation on Kolmogorov-Smirnov test.” In Proceedings of the VLDB Endowment, 14:1583–96, 2021. https://doi.org/10.14778/3461535.3461546.
Cong Z, Chu L, Yang Y, Pei J. Comprehensible counterfactual explanation on Kolmogorov-Smirnov test. In: Proceedings of the VLDB Endowment. 2021. p. 1583–96.
Cong, Z., et al. “Comprehensible counterfactual explanation on Kolmogorov-Smirnov test.” Proceedings of the VLDB Endowment, vol. 14, no. 9, 2021, pp. 1583–96. Scopus, doi:10.14778/3461535.3461546.
Cong Z, Chu L, Yang Y, Pei J. Comprehensible counterfactual explanation on Kolmogorov-Smirnov test. Proceedings of the VLDB Endowment. 2021. p. 1583–1596.

Published In

Proceedings of the VLDB Endowment

DOI

EISSN

2150-8097

Publication Date

January 1, 2021

Volume

14

Issue

9

Start / End Page

1583 / 1596

Related Subject Headings

  • 4605 Data management and data science
  • 0807 Library and Information Studies
  • 0806 Information Systems
  • 0802 Computation Theory and Mathematics