Course Launching Spring 2026 — This website is under active development. Content and materials are being continuously updated.

Learning Projects

Project Overview

These four hands-on projects are designed to build practical RL skills progressively. Each project explores different aspects of reinforcement learning, from game environments to language models and robotics. Self-paced learners are encouraged to complete all projects to gain comprehensive understanding of modern RL applications.

Project 1: Break the Agent Challenge

Recommended completion: After Module 5

Train and evaluate a deep RL agent (e.g., PPO or DQN) in a stochastic, procedurally generated environment. Identify a failure mode (reward hacking, poor generalization, adversarial exploitation) and propose a mitigation strategy.

Key Objectives:

  • • Implement and train a deep RL agent (PPO or DQN)
  • • Evaluate performance in stochastic environments
  • • Identify and document failure modes
  • • Propose mitigation strategies

Project 2: Taming the Language Model

Recommended completion: After Module 8

Using HuggingFace's TRL library or DeepSpeed-Chat, perform an RLHF loop on a toy LLM. Collect preference data, train a reward model, and apply PPO for alignment. Analyze misalignment or overoptimization.

Key Objectives:

  • • Use HuggingFace TRL or similar frameworks
  • • Collect preference data and train reward models
  • • Apply PPO for language model alignment
  • • Analyze alignment challenges

Project 3: RL in the Physical World

Recommended completion: After Module 11

Use a physical or simulated embodiment platform (e.g., PyBullet/MuJoCo) for tasks like grasping, navigation, or balance. Bonus for sim2real transfer.

Key Objectives:

  • • Work with physics simulators (PyBullet/MuJoCo)
  • • Implement control algorithms for robotic tasks
  • • Explore sim-to-real transfer challenges
  • • Document performance metrics

Project 4: RL for Impact

Recommended completion: After Module 14

Select an applied use case in a structured domain (e.g., healthcare, fintech, sustainability). Justify the use of RL, implement a prototype, and reflect on alignment/safety concerns.

Key Objectives:

  • • Select a real-world application domain
  • • Justify the use of RL for the problem
  • • Implement and evaluate a prototype
  • • Address safety and alignment concerns

Project Structure Guidelines

  • • Create a GitHub repository for your projects
  • • Include clear documentation and README files
  • • Provide requirements.txt or environment.yml for dependencies
  • • Document your experiments and findings
  • • Follow clean code practices with proper comments

Learning Guidelines

These projects are designed for self-directed learning. You're encouraged to explore, experiment, and learn from the community. When using AI assistants (e.g., ChatGPT, Copilot), document their use to track your learning process.

Remember: The goal is understanding, not just completion. Take time to experiment with different approaches and understand why certain methods work better than others.

Getting Help

If you get stuck, consider these resources: review the recommended readings, explore the provided documentation links, engage with the RL community forums, or experiment with simpler versions of the problem first. Learning RL is challenging but rewarding – persistence is key!