Privacy-preserving PLDA speaker verification using outsourced secure computation

Treiber, Amos; Nautsch, Andreas; Kolberg, Jascha; Schneider, Thomas; Busch, Christoph
Speech Communication, 1 October 2019

The usage of biometric recognition has become prevalent in various verification processes, ranging from unlocking mobile devices to verifying bank transactions. Automatic speaker verification (ASV) allows an individual to verify its identity towards an online service provider by comparing freshly sampled speech data to reference information stored on the service provider’s server. Due to the sensitive nature of biometric data, the storage and usage thereof is subject to recent EU regulations introduced as means to protect the privacy of individuals enrolled in an automatic biometric verification system. Stored biometric data need to be unlinkable, irreversible, and renewable to satisfy international standards. Preserving privacy in ASV is also important because, contrary to other biometric characteristics such as fingerprints, speech data can be used to infer a lot of sensitive information about the data subject. As a result, some architectures have been proposed to enable privacy-preserving ASV in the encrypted domain. Recently, homomorphic encryption (HE) was proposed to protect both subject features and vendor models in an embedding-based ASV. This architecture improves on previous privacy-preserving ASV by using (probabilistic) embeddings (i-vectors) and by additionally protecting the vendor’s model. However, the usage of HE comes with a rather heavy overhead and significantly slows down the verification process.

In this work, we align the cryptographic notion of outsourced secure two-party computation to embedding-based ASV. Our architecture protects biometric information in ASV and can also be used for any automatic biometric verification task. We show that unlinkability, irreversibility, and renewability are granted. Compared to the HE solution, our architecture results in considerably lower communication and computation overhead. Our architecture has been implemented and is experimentally evaluated on the NIST i-vector challenge 2014 using the cosine distance and log-likelihood ratio (LLR) scores from probabilistic linear discriminant analysis (PLDA) and two-covariance (2Cov) comparators. The results show that verification accuracy is retained while efficiency is improved. For instance, a PLDA verification with an embedding dimension of 200 takes about 77 milliseconds over a LAN. This is an improvement of more than 3000 ×  over the HE-based solution and shows that privacy of subject and vendor data can be preserved in ASV while retaining practical verification times. Moreover, our system is secure against malicious client devices.

Digital Security
Eurecom Ref:
© Elsevier. Personal use of this material is permitted. The definitive version of this paper was published in Speech Communication, 1 October 2019 and is available at :
See also: