We now live in an era of Artificial Intelligence (AI) where we rely on responses as well as actions by numerous autonomous systems that are crisscrossed in our daily lives. These systems are powered by AI that learn to provide us with reasonable answers for us with respect to our respective perspectives. Reinforcement learning is one of the most advanced and powerful way of developing such systems and are very much in line with the learning paradigms used to make us knowledgeable since our childhood, which is to learn from our mistakes. In this course, students are going to get a solid foundation in the field of reinforcement learning, learn the core challenges, and ideas to bring in newer approaches to make the systems robust, and more humanoid, and better to some degree. Through a combination of lectures, programming assignments students are expected to receive a hands-on-experience in exploring this field effectively. In addition, through the final project in this course, students will advance their understanding of reinforcement learning paradigm and are going to be able to design, develop and demonstrate by the end of the semester smart competitive players in video games, autonomous chatbots, autonomous vehicle control systems, early detection of malicious activities in the communication networks in the field of cybersecurity, and so on.

Course objectives

By the end of the course you are expected to gain the following skills:

1. learn key ideas and algorithms of reinforcement learning – a more powerful paradigm in the field of machine learning.
2. be able to understand where reinforcement learning algorithms fit to solve problems.
3. apply reinforcement learning algorithms to solve various practical problems.

Prerequisites

For undergraduate students:

1. MATH-3195 (Linear Algebra and Differential Equations) or equivalent,
2. CSCI-3412 (Algorithms) or equivalent.

For graduate students:

1. The graduate standing.

Topics covered

1. Introduction to Reinforcement Learning
2. Introduction to Amazon AWS Deepracer
3. Policy based and value based learning algorithms
4. Monte-carlo methods to learn
5. Deep Learning introduction
6. Comparative analysis of 3 reinforcement learning paradigms: DP, MC, TD
7. The SARSA algorithm
8. A non-tabular (e.g., approximation) approach in RL
9. Continous action space
10. Actor-critic
11. Value function approximation
12. On Temporal differende learning – TD(Lamba)