AIPI 531: Introduction to Modern Reinforcement Learning

Course Description

This course provides a comprehensive introduction to modern reinforcement learning, covering both theoretical foundations and practical applications. Students will learn classical algorithms like Q-learning and policy gradients, as well as modern deep RL methods including DQN, PPO, and SAC.

Special emphasis is placed on contemporary applications including RLHF for language models, game-playing agents (AlphaZero), robotics control, and safety-critical systems. Through hands-on programming assignments and a final project, students will gain practical experience implementing and training RL agents.

Learning Objectives

By the end of this course, students will be able to:

• • Formulate real-world problems as Markov Decision Processes (MDPs)
• • Implement and analyze classical RL algorithms (Q-learning, SARSA, policy gradient)
• • Design and train deep RL agents using modern frameworks
• • Apply RL to domains including robotics, NLP, and game playing
• • Understand safety, alignment, and ethical considerations in deployed RL systems
• • Read and critically evaluate current RL research papers

Prerequisites

• • Calculus, Linear Algebra, Probability & Statistics
• • Python Programming Proficiency
• • Deep Learning Applications (AIPI 540) or equivalent

Course Structure

• • 14 comprehensive modules from RL fundamentals to advanced topics
• • 4 hands-on projects with real-world applications
• • Interactive lab exercises and coding tutorials
• • Curated readings from Sutton & Barto and recent research papers