Optimization strategies for multi-task learning algorithms

Zuluaga, Maria A
Invited talk at Sophia Summit 2021, 17-19 Novembre 2021, Sophia Antipolis, France

mso-fareast-font-family:"Times New Roman";mso-ansi-language:FR;mso-fareast-language:
EN-US;mso-bidi-language:AR-SA">Multi-Task Learning (MTL) is a learning parameter involving the joint optimization of parameters with respect to multiple tasks. It is particularly appealing in small data size regimes because it offers the possibility to learn more generalizable features from the potentially limited data at disposal. In practice, however, the broad use of MTL is hindered by the lack of consistent performance gains observed by deep multi-task networks. Performance degradation in deep MTL networks has been partially explained by task interference. This phenomenon results from the destructive interactions among back-propagated gradients from the different tasks, and prevents the network from strengthening its inductive bias. In this talk, I will be discussing a novel set of optimization techniques for multi-task learners to address task interference by allowing the tasks to specify their own usage of network neurons. Differently from standard MTL optimization schemes, which aggregate all task specific objectives under a unique multi-task objective function using gradient descent methods, the proposed scheme uses independent gradient descent steps that are alternated along the different task-specific objectives. To demonstrate the generalization properties of the proposed techniques, I will present the results obtained on different state-of-the-art multi-task baselines and will conclude the talk with a real world use-case involving glaucoma diagnosis from retinal fundus images.

Sophia Antipolis
Data Science
Eurecom Ref:
© EURECOM. Personal use of this material is permitted. The definitive version of this paper was published in Invited talk at Sophia Summit 2021, 17-19 Novembre 2021, Sophia Antipolis, France and is available at :
See also:

PERMALINK : https://www.eurecom.fr/publication/6723