Ecole d'ingénieur et centre de recherche en Sciences du numérique

Efficient and self-balanced ROLLUP aggregates for large-scale data summarization

Phan, Duy-Hung; Hoang-Xuan, Quang-Nhat; Dell'Amico, Matteo; Michiardi, Pietro

BIGDATA 2015, 4th International IEEE Congress on Big Data, June 27-July 2, 2015, New York, USA

Data summarization queries that compute aggregates by grouping datasets across several dimensions are essential to help users make sense of very large datasets. In this work, we focus on ROLLUP, an important operator that has been recently added to the Hadoop MapReduce ecosystem. However, its current implementation suffers from very large communication costs, leading to inefficient executions. We thus proceed with the design of a new ROLLUP operator for highlevel languages. Our operator is self-optimizing, which means that it automatically performs load-balancing and determines a suitable operating point to achieve the highest performance. We have implemented our ROLLUP operator for Apache Pig, a popular high-level language in the Hadoop ecosystem. Our experimental results, obtained on both synthetic and real datasets, indicate that our new operator outperforms the current ROLLUP implementation in Pig by at least 50%.

Document Doi Bibtex

Titre:Efficient and self-balanced ROLLUP aggregates for large-scale data summarization
Département:Data Science
Eurecom ref:4590
Copyright: © 2015 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+4590, doi = {}, year = {2015}, title = {{E}fficient and self-balanced {ROLLUP} aggregates for large-scale data summarization}, author = {{P}han, {D}uy-{H}ung and {H}oang-{X}uan, {Q}uang-{N}hat and {D}ell'{A}mico, {M}atteo and {M}ichiardi, {P}ietro}, booktitle = {{BIGDATA} 2015, 4th {I}nternational {IEEE} {C}ongress on {B}ig {D}ata, {J}une 27-{J}uly 2, 2015, {N}ew {Y}ork, {USA}}, address = {{N}ew-{Y}ork, {\'{E}}{TATS}-{UNIS}}, month = {06}, url = {} }
Voir aussi: