Differential privacy

Adversarial attacks aim to deceive ML systems by leading them to make wrong decisions, for instance by learning necessary information about a classifier, by directly modifying the model or misclassifying inputs. Adversarial ML [1,2] studies these attacks and defenses created against them. Introducing adversarial examples to ML systems is a specific type of sophisticated and powerful attack, where additional and sometimes specially crafted or modified inputs are provided to the system with the intent of being misclassified by the model as legitimate as in the case of misclassification attacks [2] and the adversarial classifier reverse engineer learning problem [3]. Another class of adversarial attacks is constructed to infer membership [4-5], where the adversary’s goal is to decide whether a given data sample was included in the training dataset of the targeted ML model. A common solution that may be tailored to counter each of these different types of adversarial attacks is offered by differential privacy (DP) [6], which is a stochastic measure of privacy and is now used in conjunction with ML algorithms to guarantee privacy of individual users while handling large datasets. DP has furthermore been used to develop practical methods for protecting private user-data at the moment they provide information to the ML system. In this case, the use of a differentially private measure aims to maintain the accuracy of the ML model without incurring a cost of the privacy of individual participants. A mechanism is said to be differentially private if the level of privacy of its users and the output of the mechanism remain unaltered, even when any of the participants decides to submit or remove their personal information from the statistical dataset. This tutorial delivers an extensive summary on the theory of DP along with its properties as well as some examples of its use in practice to shield a chosen set of ML algorithms from a number of different adversarial attacks.