Graduate School and Research Center in Digital Sciences

User-driven error detection for time series with events

Le, Kim-Hung; Papotti, Paolo

ICDE 2020, 36th IEEE International Conference on Data Engineering, 20-24 April, Dallas, Texas, USA

Anomalies are pervasive in time series data, such as sensor readings. Existing methods for anomaly detection cannot distinguish between anomalies that represent data errors, such as incorrect sensor readings, and notable events, such as the watering action in soil monitoring. In addition, the quality performance of such detection methods highly depends on the configuration parameters, which are dataset specific. In this work, we exploit active learning to detect both errors and events in a single solution that aims at minimizing user interaction. For this joint detection, we introduce an algorithm that accurately detects and labels anomalies with a non-parametric concept of neighborhood and probabilistic classification. Given a desired quality, the confidence of the classification is then used as termination condition for the active learning algorithm. Experiments on real and synthetic datasets demonstrate that our approach achieves F-score above 80% in detecting errors by labeling 2 to 5 points in one data series. We also show the superiority of our solution compared to the state-of-the-art approaches for anomaly detection. Finally, we demonstrate the positive impact of our error detection methods in downstream data repairing algorithms.

Document Doi Bibtex

Title:User-driven error detection for time series with events
Type:Conference
Language:English
City:Dallas
Country:UNITED STATES
Date:
Department:Data Science
Eurecom ref:6192
Copyright: © 2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+6192, doi = {http://dx.doi.org/10.1109/ICDE48307.2020.00070}, year = {2020}, title = {{U}ser-driven error detection for time series with events}, author = {{L}e, {K}im-{H}ung and {P}apotti, {P}aolo}, booktitle = {{ICDE} 2020, 36th {IEEE} {I}nternational {C}onference on {D}ata {E}ngineering, 20-24 {A}pril, {D}allas, {T}exas, {USA}}, address = {{D}allas, {UNITED} {STATES}}, month = {04}, url = {http://www.eurecom.fr/publication/6192} }
See also: