Graduate School and Research Center in Digital Sciences

OS-assisted task preemption for Hadoop

Pastorelli, Mario; Dell'Amico, Matteo; Michiardi, Pietro

DCPERF 2014, 4th International Workshop on Data Center Performance, June 30-July 3rd, 2014, Madrid, Spain / Also published on ArXiv

This work introduces a new task preemption primitive for Hadoop, that allows tasks to be suspended and resumed exploiting existing memory management mechanisms readily available in modern operating systems. Our technique fills the gap that exists between the two extremes cases of killing tasks (which waste work) or waiting for their completion (which introduces latency): experimental results indicate superior performance and very small overheads when compared to existing alternatives.

Document Doi Arxiv Bibtex

Title:OS-assisted task preemption for Hadoop
Department:Data Science
Eurecom ref:4223
Copyright: © 2014 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+4223, doi = {}, year = {2014}, title = {{OS}-assisted task preemption for {H}adoop}, author = {{P}astorelli, {M}ario and {D}ell'{A}mico, {M}atteo and {M}ichiardi, {P}ietro}, booktitle = {{DCPERF} 2014, 4th {I}nternational {W}orkshop on {D}ata {C}enter {P}erformance, {J}une 30-{J}uly 3rd, 2014, {M}adrid, {S}pain / {A}lso published on {A}r{X}iv}, address = {{M}adrid, {SPAIN}}, month = {02}, url = {} }
See also: