Abstract:
The purpose of this course is to introduce the student to new and exciting applications whose analysis and resolution will employ powerful yet simple information-theoretic tools. Such applications will be in the context of communications, distributed computing, compressive sensing as well as data science.
Teaching and Learning Methods
The course will cover material from a variety of seminal textbooks, and will try to instill a proper mix of theory and practice. The student will be taught a variety of information theoretic approaches, and will be exposed to the technological impact of these approaches. We will learn how basic mathematical machinery can be used to provide clear insight and simple solutions to some of the most interesting technological problems in communications, distributed computing, compressive sensing and machine learning.
Course Policies
- Exams: The (final) exam will be two hours long, and it will be comprehensive. During the exam, all notes from this taught class are allowed.
- Project: The work performed during the project can be collaborative.
- Elements of Information Theory, by T. M. Cover and J. A. Thomas, Wiley Series.
- Network Information Theory, by A. El Gamal and Y. Kim, Cambridge University Press.
- Fundamentals of Wireless Communication, by D. Tse and P. Viswanath, Cambridge University Press.
- Information Theory Level 1 (strongly recommended but not compulsory)
- An understanding of probability theory and random
Description
The basic structure of the course is as follows. First it will involve a basic study of various
-
Information Theoretic Metrics, that include:
- Channel Capacity
- Degrees of Freedom
- e-Outage Capacity
- Reliability Metrics.
Then the course will place emphasis on communication-related topics, by studying various
-
Classical Channel Settings, such as:
- Multiple-Access Channels (Various performance regions)
- Class of Broadcast Channels
- Various Relay Channels (Capacity and outage results).
Making the transition to data science, where indeed the nature of data is of crucial importance, the course will then involve the study of various
-
Classical Source Settings, such as:
- The Slepian-Wolf approach
- Lossy Source Coding
- Semantic considerations.
Subsequently the course will jointly explore communications and distributed computing by studying
- Advanced Network Information Theory, focusing on topics such as:
- Network Coding
- Index Coding and Caching
- Coded Distributed Computing.
Finally, motivated by their sheer technological impact, the course will explore, time permitting, topics that involve various
- Information-theoretic Applications to Federated Learning and Compressed Sensing.
Learning outcomes
The main learning goal of this course will be to help the student employ simple mathematical tools stemming from information theory and statistics, in order to analyze and resolve various modern technological challenges in a variety of areas.
Nb hours: 21 hours
Evaluation
- Final exam (60% of the final grade)
- Collaborative Project (40%of the final grade)
