Lucas Pascal (EURECOM, Data Science Department) - Cifre PhD Data Science
Date: - Location: Eurecom
Abstract: Multi-task learning has gained popularity due to the advantages it provides with respect to resource usage and performance. Nonetheless, the joint optimization of parameters with respect to multiple tasks remains an active research topic. Sub-partitioning the parameters between different tasks has proven to be an efficient way to relax the optimization constraints over the shared weights, may the partitions be disjoint or overlapping. However, one drawback of this approach is that it can weaken the inductive bias generally set up by the joint task optimization. In this work, we present a novel way to partition the parameter space without weakening the inductive bias. Specifically, we propose Maximum Roaming, a method inspired by dropout that randomly varies the parameter partitioning, while forcing them to visit as many tasks as possible at a regulated frequency, so that the network fully adapts to each update. We study the properties of our method through experiments on a variety of visual multi-task data sets. Experimental results suggest that the regularization brought by roaming has more impact on performance than usual partitioning optimization strategies. The overall method is flexible, easily applicable, provides superior regularization and consistently achieves improved performances compared to recent multi-task learning formulations. Reference: Pascal, L., Michiardi, P., Bost, X., Huet, B. & Zuluaga, M.A. (2021). Maximum Roaming Multi-Task Learning. AAAI 2021. https://arxiv.org/abs/2006.09762 Biography: Lucas is a Cifre PhD candidate under the supervision of successively Benoit Huet (2018-2019), Pietro Michiardi and Maria Zuluaga (2020-2021) with EURECOM and Orkis (Aix-en-Provence, France), a small company which provides a Digital Asset Management solution for professional customers. He holds an Engineer diploma in Computer Science and Applied Mathematics from ENSEEIHT and a MSc in multimedia computing, both from the ENSEEIHT engineering school (Toulouse, France). In 2017 he was in Technicolor (Rennes, France), working on a Color Grading replication tool based on sparse user constraints for cinema industry. His main research is now focused on novel solutions to improve the optimization of deep multi-task neural networks for vision tasks. In the past few years, he gave talks and poster presentations in CBMI 2019 (Dublin, IRL), MICCAI 2020 for the Ophthalmic Medical Image Analysis (OMIA) workshop and AAAI 2021 (both virtual). Data Science Seminars: https://ds.eurecom.fr/seminars/
