Graduate School and Research Center in Digital Sciences

Deep Learning

[DeepLearning]
T Technical Teaching


Abstract

Deep Learning is a new approach in Machine Learning which allows to build models that have shown superior performance fora wide range of applications, in particular Computer Vision and Natural Language Processing. Thanks to the joint availability of large data corpus and affordable processing power, Deep Learning has revived the old field  of Artificial Neural Networks and provoked the "Renaissance" of AI (Artificial Intelligence). The objective of this course is to provide an overview of the field of Deep Learning, starting from simple Neural Network architectures and pursuing with contemporary and state of the art practices and models. The course is organized as a combination of  lectures where the theory is exposed and discussed, and hands-on sessions (labs) where experimentations are performed to practice with the theoretical concepts.

Teaching and Learning Methods : The course is composed of a combination of lectures and labs.

Course Policies : Attendance to all sessions is mandatory.

Bibliography

Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016). Deep Learning. MIT Press.

Deng, L.; Yu, D. (2014). "Deep Learning: Methods and Applications". Foundations and Trends in Signal Processing. 7 (3-4): 1-199. doi:10.1561/2000000039.

Bengio, Yoshua; LeCun, Yann; Hinton, Geoffrey (2015). "Deep Learning". Nature. 521: 436-444. doi:10.1038/nature14539

Requirements

Basic knowledge in Data Science

Description

The course is intended to expose students with the recent developments in the field of Deep Artificial Neural Networks (DANN). It will start with a brief reminder about the theory of Neural Networks and then move on to present the evolution of DANN introduced in the recent years.

The study will  start with the Multilayer Feed Forward (Multilayer Perceptron) architecture and the famous "Back Propagation" algorithms along with its various parameters and variants as they provide the basis for most recent advances. Then the course will explore the specific problems of Deep Networks, and some of the solutions that have been proposed to overcome them, such as Drop-out and Rectified Linear Unit (ReLU). This will include unsupervised approach using auto-encoders. Then it will describe specific architectures involved in Deep Networks, such as Convolutional Neural Networks (CNNs) for image processing, Residual Networks which allow very deep networks, Recurrent Neural Networks (RNN) for language modeling and Long Short Term Memory (LSTM) for learning long-term dependencies. The course will present and detail the specific techniques which are involved in their applications, such as transfer learning and fine tuning for CNNs, backpropagation through time, together with the issues which appear in their implementation and practical usage.

Throughout the course, applications of the various network topologies are discussed in order to illustrate their usage. The labs will use some of the standard development environments for DNN, such as Caffe and Tensorflow.

Learning outcomes:

-      be able to understand the key fundamentals associated with Deep Learning and Deep Network architectures for Machine Learning.

-      be able to define, train and use a Deep Neural Network.

Grading Policy:: Labs and written exam. 

Nb hours: 21.00
Nb hours per week: 3.00