Loading...
DyScale: A MapReduce Job Scheduler for Heterogeneous Multicore Processors
Yan, Feng Cherkasova, Ludmila Zhang, Zhuoyao Smirni, Evgenia
Yan, Feng
Cherkasova, Ludmila
Zhang, Zhuoyao
Smirni, Evgenia
Abstract
The functionality of modern multi-core processors is often driven by a given power budget that requires designers to evaluate different decision trade-offs, e.g., to choose between many slow, power-efficient cores, or fewer faster, power-hungry cores, or a combination of them. Here, we prototype and evaluate a new Hadoop scheduler, called DyScale, that exploits capabilities offered by heterogeneous cores within a single multi-core processor for achieving a variety of performance objectives. A typical MapReduce workload contains jobs with different performance goals: large, batch jobs that are throughput oriented, and smaller interactive jobs that are response time sensitive. Heterogeneous multi-core processors enable creating virtual resource pools based on "slow" and "fast" cores for multi-class priority scheduling. Since the same data can be accessed with either "slow" or "fast" slots, spare resources (slots) can be shared between different resource pools. Using measurements on an actual experimental setting and via simulation, we argue in favor of heterogeneous multi-core processors as they achieve "faster" (up to 40 percent) processing of small, interactive MapReduce jobs, while offering improved throughput (up to 40 percent) for large, batch jobs. We evaluate the performance benefits of DyScale versus the FIFO and Capacity job schedulers that are broadly used in the Hadoop community.
Description
Date
2017-07-01
Journal Title
Journal ISSN
Volume Title
Publisher
Collections
Download Dataset
Files
Loading...
DyScale.pdf
Adobe PDF, 1.4 MB
Rights Holder
Usage License
Embargo
Research Projects
Organizational Units
Journal Issue
Keywords
Citation
Advisor
Department
Computer Science
DOI
https://doi.org/10.1109/TCC.2015.2415772