EURECOM at SemEval-2024 Task 4: Hierarchical Loss and Model Ensembling in Detecting Persuasion Techniques

Peskine, Youri; Troncy, Raphaël; Papotti, Paolo
SEMEVAL 2024, 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024, 20-21 June 2024, Mexico City, Mexico

This paper describes the submission of team EURECOM at SemEval-2024 Task 4: Multilingual Detection of Persuasion Techniques in Memes. We only tackled the first sub-task, consisting of detecting 20 named persuasion techniques in the textual content of memes. We trained multiple BERT-based models (BERT, RoBERTa, BERT pre-trained on harmful detection) using different losses (Cross Entropy, Binary Cross Entropy, Focal Loss and a custommade hierarchical loss). The best results were obtained by leveraging the hierarchical nature of the data, by outputting ancestor classes and with a hierarchical loss. Our final submission consist of an ensembling of our top-3 best models for each persuasion techniques. We obtain hierarchical F1 scores of 0.655 (English), 0.345 (Bulgarian), 0.442 (North Macedonian) and 0.177 (Arabic) on the test set.


HAL
Type:
Conference
City:
Mexico City
Date:
2024-06-18
Department:
Data Science
Eurecom Ref:
7779
Copyright:
Copyright ACL. Personal use of this material is permitted. The definitive version of this paper was published in SEMEVAL 2024, 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024, 20-21 June 2024, Mexico City, Mexico and is available at :

PERMALINK : https://www.eurecom.fr/publication/7779