Toward automatic management of embarrassingly parallel applications
Publication
, Journal Article
Dutra, I; Page, D; Costa, VS; Shavlik, J; Waddell, M
Published in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
December 1, 2004
Large-scale applications that require executing very large numbers of tasks are only feasible through parallelism. In this work we present a system that automatically handles large numbers of experiments and data in the context of machine learning. Our system controls all experiments, including re-submission of failed jobs and relies on available resource managers to spawn jobs through pools of machines. Our results show that we can manage a very large number of experiments, using a reasonable amount of idle CPU cycles, with very little user intervention. © Springer-Verlag 2003.
Duke Scholars
Published In
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
EISSN
1611-3349
ISSN
0302-9743
Publication Date
December 1, 2004
Volume
2790
Start / End Page
509 / 516
Related Subject Headings
- Artificial Intelligence & Image Processing
Citation
APA
Chicago
ICMJE
MLA
NLM
Dutra, I., Page, D., Costa, V. S., Shavlik, J., & Waddell, M. (2004). Toward automatic management of embarrassingly parallel applications. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2790, 509–516.
Dutra, I., D. Page, V. S. Costa, J. Shavlik, and M. Waddell. “Toward automatic management of embarrassingly parallel applications.” Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2790 (December 1, 2004): 509–16.
Dutra I, Page D, Costa VS, Shavlik J, Waddell M. Toward automatic management of embarrassingly parallel applications. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2004 Dec 1;2790:509–16.
Dutra, I., et al. “Toward automatic management of embarrassingly parallel applications.” Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2790, Dec. 2004, pp. 509–16.
Dutra I, Page D, Costa VS, Shavlik J, Waddell M. Toward automatic management of embarrassingly parallel applications. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2004 Dec 1;2790:509–516.
Published In
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
EISSN
1611-3349
ISSN
0302-9743
Publication Date
December 1, 2004
Volume
2790
Start / End Page
509 / 516
Related Subject Headings
- Artificial Intelligence & Image Processing