Document Type
Article
Department/Program
Computer Science
Journal Title
IEEE TRANSACTIONS ON CLOUD COMPUTING
Pub Date
7-2017
Volume
5
Issue
2
Abstract
The functionality of modern multi-core processors is often driven by a given power budget that requires designers to evaluate different decision trade-offs, e.g., to choose between many slow, power-efficient cores, or fewer faster, power-hungry cores, or a combination of them. Here, we prototype and evaluate a new Hadoop scheduler, called DyScale, that exploits capabilities offered by heterogeneous cores within a single multi-core processor for achieving a variety of performance objectives. A typical MapReduce workload contains jobs with different performance goals: large, batch jobs that are throughput oriented, and smaller interactive jobs that are response time sensitive. Heterogeneous multi-core processors enable creating virtual resource pools based on "slow" and "fast" cores for multi-class priority scheduling. Since the same data can be accessed with either "slow" or "fast" slots, spare resources (slots) can be shared between different resource pools. Using measurements on an actual experimental setting and via simulation, we argue in favor of heterogeneous multi-core processors as they achieve "faster" (up to 40 percent) processing of small, interactive MapReduce jobs, while offering improved throughput (up to 40 percent) for large, batch jobs. We evaluate the performance benefits of DyScale versus the FIFO and Capacity job schedulers that are broadly used in the Hadoop community.
Recommended Citation
Yan, Feng; Cherkasova, Ludmila; Zhang, Zhuoyao; and Smirni, Evgenia, DyScale: A MapReduce Job Scheduler for Heterogeneous Multicore Processors (2017). IEEE TRANSACTIONS ON CLOUD COMPUTING, 5(2).
10.1109/TCC.2015.2415772
DOI
10.1109/TCC.2015.2415772
