Ecole d'ingénieur et centre de recherche en Sciences du numérique

t-DCF: a detection cost function for the tandem assessment of spoofing countermeasures and automatic speaker verification

Kinnunen, Tomi; Lee, Kong Aik; Delgado, Héctor; Evans, Nicholas; Todisco, Massimiliano; Sahidullah, Md; Yamagishi, Junichi; Reynolds, Douglas A.

ODYSSEY 2018, The Speaker and Language Recognition Workshop, June 26-29, 2018, Les Sables d'Olonne, France

The ASVspoof challenge series was born to spearhead research in anti-spoofing for automatic speaker verification (ASV). The two challenge editions in 2015 and 2017 involved the assessment of spoofing countermeasures (CMs) in isolation from ASV using an equal error rate (EER) metric. While a strategic approach to assessment at the time, it has certain shortcomings. First, the CM EER is not necessarily a reliable predictor of performance when ASV and CMs are combined. Second, the EER operating point is ill-suited to user authentication applications, e.g. telephone banking, characterised by a high target user prior but a low spoofing attack prior. We aim to migrate from CM- to ASV-centric assessment with the aid of a new tandem detection cost function (t-DCF) metric. It extends the conventional DCF used in ASV research to scenarios involving spoofing attacks. The t-DCF metric has 6 parameters: (i) false alarm and miss costs for both systems, and (ii) prior probabilities of target and spoof trials (with an implied third, nontarget prior). The study is intended to serve as a self-contained, tutorial-like presentation. We analyse with the t-DCF a selection of top-performing CM submissions to the 2015 and 2017 editions of ASVspoof, with a focus on the spoofing attack prior. Whereas there is little to choose between countermeasure systems for lower priors, system rankings derived with the EER and t-DCF show differences for higher priors. We observe some ranking changes. Findings support the adoption of the DCFbased metric into the roadmap for future ASVspoof challenges, and possibly for other biometric anti-spoofing evaluations. 

Document Hal Bibtex

Titre:t-DCF: a detection cost function for the tandem assessment of spoofing countermeasures and automatic speaker verification
Type:Conférence
Langue:English
Ville:Les Sables d'Olonne
Pays:FRANCE
Date:
Département:Sécurité numérique
Eurecom ref:5524
Copyright: © ISCA. Personal use of this material is permitted. The definitive version of this paper was published in ODYSSEY 2018, The Speaker and Language Recognition Workshop, June 26-29, 2018, Les Sables d'Olonne, France and is available at :
Bibtex: @inproceedings{EURECOM+5524, year = {2018}, title = {t-{DCF}: a detection cost function for the tandem assessment of spoofing countermeasures and automatic speaker verification}, author = {{K}innunen, {T}omi and {L}ee, {K}ong {A}ik and {D}elgado, {H}{\'e}ctor and {E}vans, {N}icholas and {T}odisco, {M}assimiliano and {S}ahidullah, {M}d and {Y}amagishi, {J}unichi and {R}eynolds, {D}ouglas {A}.}, booktitle = {{ODYSSEY} 2018, {T}he {S}peaker and {L}anguage {R}ecognition {W}orkshop, {J}une 26-29, 2018, {L}es {S}ables d'{O}lonne, {F}rance }, address = {{L}es {S}ables d'{O}lonne, {FRANCE}}, month = {06}, url = {http://www.eurecom.fr/publication/5524} }
Voir aussi: