Revealing the unknown ADSL traffic using statistical methods

Traffic classification is one of the most significant issues for ISPs and  network administrators. Recent research on the subject resulted in a large variety  of algorithms and methods applicable to the problem. In this work, we focus  on several issues that have not received enough attention so far. First, the establishment of an accurate reference point. We use an ISP internal Deep Packet Inspection (DPI) tool and confront its results with state of the art, freely available classification tools, finding significant differences. We relate those differences to the weakness of some signatures and to the heuristics and design choices made by DPI tools. Second, we highlight methodological issues behind the choices of  the traffic classes and the way of analyzing the results of a statistical classifier.  Last, we focus on the often overlooked problem of mining the unknown traffic, i.e., traffic not classified by the DPI tool used to establish the reference point. We  present a method, relying on the level of confidence of the statistical classification, to reveal the unknown traffic. We further discuss the result of the classifier using a variety of heuristics.