drawing

ECE 720B01 Robot Learning: Principles and Advances (winter term 2025)

Instructors:
Professor: Jun Jin
TAs: Yi Hu, hu19@ualberta.ca

Lectures: Monday Wednesday 09:30 to 10:50, ECERF W3-087

Office Hours: Wednesdays, 11:00 - 11:50, 11-365 Donadeo Innovation Centre For Engineering

Announcements

  • Welcome!
  • This is my second year teaching this course. Thanks for your interest. This year, we will make this course more interactive!

Prerequisites

  • Familiarity with mathematical proofs, probability, algorithms, and linear algebra; ability to implement algorithmic ideas in code.
  • Key Machine Learning courses that can build your background: ECE 447, ECE 449, COMPUT 466/566. If your foundations are rusty, we recommend you refresh them through those courses.
  • All graduate students are welcome to register.

Class Goals

  • Learn the math and algorithms in modern machine learning based robotic systems.
  • Grasp an overall picture of state-of-the-art literature in robot learning, with a special focus on robotic reinforcement learning (RL), robot learning from human preference, multi-task learning and continual learning (CL) and foundation models (LLMs, decision transformers) for robotics.
  • An intended side-effect of the course is to generally strengthen your expertise in these areas.

  • Be able to understand research papers in the field of robotics:

    • Main conferences: ICRA, IROS, RSS, CoRL, NeurIPS, ICML, ISER, ISRR.
    • Main journals: IJRR, T-RO, Autonomous Robots, RA-L.
  • Try out some ideas/extensions of your own.
  • Note: the focus of this course is on math and algorithms. We will not study mechanical or electrical design of robots.

Grading

  • Assignments (20%)

    • There are only 2 assignments.
    • 1 coding task (<10 lines) + 1 writing task about your project idea.

  • Read & Present what you've learned (10%)

    • 5 min paper reading presentation.

      • Select a paper from our course reading list, and present what you have learned.

  • Think & Speak out your opinion (10%)

    • 5 min machine learning street talk.

      • Do these methods excite you, or do you feel they rely on overly simplistic or unrealistic assumptions? What problems are you curious about? Do you have better solutions in mind? And if you were given $1 million, what kind of project would you pursue?

  • Review & Contribute your advice (10%)

    • Review 1 course project report, mark your rating, and leave your review comments.

  • Final project (50%)

    • 1~2 students per project.

    • Implement a learning algorithm (e.g., imitation learning, RL) of your choice to solve any decision-making tasks (robotics, games, control, prediction, etc.).

      • If you need a starting code repo to make your project much easier, we’ll provide a code repo with (1) a robotic simulator ready for training, (2) a standard openai-gym environment ready for plug-and-play using existing opensource ML algorithm codes (e.g., the StableBaselines)

    • 1/5 quality of presentation, 1/5 quality of writing of the final paper, 3/5 quality of the results themselves.

      • Extra points will be awarded to solutions that explore more advanced topics, e.g., inverse reinforcement learning (IRL), generalization, meta-learning, motion primitives, LLMs, etc.

Syllabus and materials

All slides will be released on the website after each lecture. All slides are open to the public and free to download.