Skip to main content

Optimizing I/O for big array analytics

Publication ,  Journal Article
Zhang, Y; Yang, J
Published in: Proceedings of the VLDB Endowment
January 1, 2012

Big array analytics is becoming indispensable in answering important scientific and business questions. Most analysis tasks consist of multiple steps, each making one or multiple passes over the arrays to be analyzed and generating intermediate results. In the big data setting, I/O optimization is a key to efficient analytics. In this paper, we develop a framework and techniques for capturing a broad range of analysis tasks expressible in nested-loop forms, representing them in a declarative way, and optimizing their I/O by identifying sharing opportunities. Experiment results show that our optimizer is capable of finding execution plans that exploit nontrivial I/O sharing opportunities with significant savings. © 2012 VLDB Endowment.

Duke Scholars

Published In

Proceedings of the VLDB Endowment

DOI

EISSN

2150-8097

Publication Date

January 1, 2012

Volume

5

Issue

8

Start / End Page

764 / 775

Related Subject Headings

  • 4605 Data management and data science
  • 0807 Library and Information Studies
  • 0806 Information Systems
  • 0802 Computation Theory and Mathematics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhang, Y., & Yang, J. (2012). Optimizing I/O for big array analytics. Proceedings of the VLDB Endowment, 5(8), 764–775. https://doi.org/10.14778/2212351.2212358
Zhang, Y., and J. Yang. “Optimizing I/O for big array analytics.” Proceedings of the VLDB Endowment 5, no. 8 (January 1, 2012): 764–75. https://doi.org/10.14778/2212351.2212358.
Zhang Y, Yang J. Optimizing I/O for big array analytics. Proceedings of the VLDB Endowment. 2012 Jan 1;5(8):764–75.
Zhang, Y., and J. Yang. “Optimizing I/O for big array analytics.” Proceedings of the VLDB Endowment, vol. 5, no. 8, Jan. 2012, pp. 764–75. Scopus, doi:10.14778/2212351.2212358.
Zhang Y, Yang J. Optimizing I/O for big array analytics. Proceedings of the VLDB Endowment. 2012 Jan 1;5(8):764–775.

Published In

Proceedings of the VLDB Endowment

DOI

EISSN

2150-8097

Publication Date

January 1, 2012

Volume

5

Issue

8

Start / End Page

764 / 775

Related Subject Headings

  • 4605 Data management and data science
  • 0807 Library and Information Studies
  • 0806 Information Systems
  • 0802 Computation Theory and Mathematics