Deep Learning is a new approach in Machine Learning which allows building models that have shown superior performance for a wide range of applications, in particular Computer Vision and Natural Language Processing. Thanks to the joint availability of large data corpus and affordable processing power, Deep Learning has revived the old field of Artificial Neural Networks and provoked the "Renaissance" of AI (Artificial Intelligence). The objective of this course is to provide an overview of the field of Deep Learning, starting from simple Neural Network architectures and pursuing contemporary and state-of-the-art models. The course is organized as a combination of lectures where the theory is exposed and discussed, and hands-on sessions (labs) where experimentations are performed to practice the theoretical concepts.
Teaching and Learning Methods: The course is composed of a combination of lectures and labs.
Course Policies: Attendance at all sessions is mandatory.
- Book: GOODFELLOW I., BENGIO Y., COURVILLE A. Deep Learning. MIT Press, 2016, 800p.
- Book: DENG L., YU D. Deep Learning: Methods and Applications. Now Publishers Inc, 2014, 212p.
- Publication: BENGIO Y., LECUN Y., HINTON G. Deep Learning. Nature. 521: 436-444. doi:10.1038/nature14539, 2015.
Basic knowledge in Data Science
The course is intended to expose students to the recent developments in the field of Deep Artificial Neural Networks (DANN). It will start with a brief reminder about the theory of Neural Networks and then move on to present the evolution of DANN introduced in recent years.
The study will start with the Multilayer Feed Forward (Multilayer Perceptron) architecture and the famous "Back Propagation" algorithms along with its various parameters and variants as they provide the basis for the most recent advances. Then the course will explore the specific problems of Deep Networks, and some of the solutions that have been proposed to overcome them, such as drop-out and Rectified Linear Unit (ReLU). This will include an unsupervised approach using auto-encoders. Then it will describe specific architectures involved in Deep Networks, such as Convolutional Neural Networks (CNNs) for image processing, Residual Networks which allow very deep networks, Recurrent Neural Networks (RNN) for language modeling and Long Short-Term Memory (LSTM) for learning long-term dependencies. The course will present and detail the specific techniques which are involved in their applications, such as transfer learning fine-tuning for CNNs, and backpropagation through time, together with the issues which appear in their implementation and practical usage.
Throughout the course, applications of the various network topologies are discussed to illustrate their usage. The labs will use some of the standard development environments for DNN, such as Caffe and TensorFlow.
Learning outcomes:
- Be able to understand the key fundamentals associated with Deep Learning and Deep Network architectures for Machine Learning.
- Be able to define, train and use a Deep Neural Network.
Evaluation:
- Lab. reports,
- Written exam.
