Dr. Jun Jin, University of Alberta

GoogleScholar / CV

Jun Jin

I am an assistant professor in the ECE department at the University of Alberta and a Fellow of Alberta Machine Intelligence Institute (Amii). Generally, I am interested in Robotics and Machine Learning research. Specifically, I focus on topics in robotic reinforcement learning which intersects with embodied artificial intelligence, the theory of predictive coding, continual learning and open-ended learning agents. In applied research, I am starting projects in AI for construction (optimal and multi-modal construction planning agents, robotics for infrastructure maintenance) and healthcare robotics.

Email: jun dot jin at ualberta.ca, replace dot with . and at with @ + remove all spaces.
Office: 11-365 Donadeo ICE building, 9211-116 St, Edmonton, AB, T6G 2H5

News

Our group is hiring! Check our lab posting.
Dr. Jun Jin is selected as an Amii Fellow!
Real-world robotic RL paper is selected as Outstanding Student Paper Award Finalist at ICRA 2022 from all submission tracks!

My research roadmap

My long-term goal is to discover first-principle learning priors to develop scalable learning architectures and algorithms to make robots (agents) agile to move, easier to program, and human-aware to interact with, which in all compose my vision for the future of Robotics and Artificial Intelligence: Human-Centered Autonomy. I am mostly facinated by the idea of building an agent model that is embodied and embedded in the environment, which means internally, the agent learns extendable emobodied skills from its past experience, and externally, the agent model has interface to enable it embedded in the environment.

In a layman's languange, I aim to build an agent model that can easily adapt its learned skills to new tasks and new environmental dynamics, that can be easily re-programmed by human, and that that is human-aware to interact with. Check more details of this research roadmap here.

In applied research, I am starting projects in AI for construction (data-driven operational optimization, infrastructure robotics), and health-care robotics. I welcome students from a Civil Engineering or Biomedical Engineering background in the University of Alberta to work with me.

My research philosophy

(1) Scientific discovery: a revised "Bitter Lesson" for robotics.
I am a believer in most parts of "The Bitter Lesson" written by Dr. Rich Sutton, but robotics research has its own characteristics that are different from a general agent definition like in Atari games, which are (1) Data is more costly, and (2) Human is an important factor! I believe we should bake in some "smart" learning architectures and algorithms in somewhere middle, as depicted in the diagram below, to address the two challenges. Therefore, finding first-principle learning priors in a data-driven approach becomes important!

(2) Social impact: take a BROADER view, it's not only full-autonomy!
Tavakoli et al. 2020 provide a holistic view of robotic applications categorized by how much task uncertainty that the robot deals with. It clearly shows that the social impact of robotics can be much broader than researches only consider full-autonomy, which is what most of our peer researchers in robot learning are working on.

I propose to take a broader view, to move from only full-autonomy to human-centered autonomy, which will help us shape the future of robotics research towards building a world with robots by human side, providing every help as best depicted in the ICRA 2015 poster above.

Publications

Predictive coding accelerates real-world robotic reinforcement learning!

Offline learning of counterfactual predictions for real-world robotic reinforcement learning

J. Jin, D. Graves, C. Haigh, J. Luo, and M. Jagersand
2022 IEEE International Conference on Robotics and Automation (ICRA) Oustanding Student Paper Award Finalist ** ICRA selects only 1 outstanding paper from 3 finalists among all submission tracks.
Paper

Our social-safety aware RL algorithm enables a mobile robot navigate in dynamic human crowds.

Mapless navigation among dynamics with social-safety-awareness: a reinforcement learning approach from 2d laser scans

J. Jin, N. M. Nguyen, N. Sakib, D. Graves, H. Yao, and M. Jagersand
2020 IEEE international conference on robotics and automation (ICRA)
Paper

Projecting on predictive prototypes enables a better open-ended learning agent.

Build generally reusable agent-environment interaction models

J. Jin, H. Zhang, and J. Luo
Advances in neural information processing systems (NeurIPS) 2022 Workshop: Foundation Models for Decision Making
Paper • Slides • Poster

Except for semantic task structure, geometric task structures can be more powerful!

Generalizable task representation learning from human demonstration videos: a geometric approach

J. Jin and M. Jagersand
2022 IEEE International Conference on Robotics and Automation (ICRA)
Paper

Embodied-GPT: Vision-language pre-training via embodied chain of thought

Y. Mu, Q. Zhang, M. Hu, W. Wang, M. Ding, J. Jin, B. Wang, J. Dai, Y. Qiao, and P. Luo
Advances in neural information processing systems (NeurIPS), 2023, (**Spotlight Award**)
Webpage • Paper

Variable decision-frequency option critic

A. Karimi, J. Jin, J. Luo, A. R. Mahmood, M. Jagersand, and S. Tosatto
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Paper

Auxiliary task discovery through generate and test

B. Rafiee, S. Ghiassian, J. Jin, R. Sutton, J. Luo, and A. White
Second Conference on Lifelong Learning Agents (CoLLAs), 2023
Paper

Replay memory as an empirical mdp: Combining conservative estimation with experience replay

H. Zhang, C. Xiao, H. Wang, J. Jin, B. Xu, and M. Muller
International Conference on Learning Representations (ICLR), 2023
Paper

Offline learning of counterfactual predictions for real-world robotic reinforcement learning

J. Jin, D. Graves, C. Haigh, J. Luo, and M. Jagersand
2022 IEEE International Conference on Robotics and Automation (ICRA)
Oustanding Student Paper Award Finalist ** ICRA selects only 1 outstanding paper from 3 finalists among all submission tracks.
Paper

Generalizable task representation learning from human demonstration videos: a geometric approach

J. Jin and M. Jagersand
2022 IEEE International Conference on Robotics and Automation (ICRA)
Paper

A quantitative analysis of activities of daily living: Insights into improving functional independence with assistive robotics

L. Petrich, J. Jin, M. Dehghan, and M. Jagersand
2022 IEEE International Conference on Robotics and Automation (ICRA)
Paper

Build generally reusable agent-environment interaction models

J. Jin, H. Zhang, and J. Luo
Advances in neural information processing systems (NeurIPS) 2022 Workshop: Foundation Models for Decision Making
Paper • Slides • Poster

A simple decentralized cross-entropy method

Z. Zhang, J. Jin, M. Jagersand, J. Luo, and D. Schuurmans
Advances in neural information processing systems (NeurIPS), 2022
Paper • Code

What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy

B. Rafiee, J. Jin, J. Luo, and A. White
The 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making, RLDM, 2022
Paper

Learning robust driving policies without online exploration

D. Graves, N. M. Nguyen, K. Hassanzadeh, J. Jin, and J. Luo
2021 IEEE International Conference on Robotics and Automation (ICRA)
Paper

A generative model-based predictive display for robotic teleoperation

B. Xie, M. Han, J. Jin, M. Barczyk, and M. Jagersand
2021 IEEE International Conference on Robotics and Automation (ICRA)
Paper

Mapless navigation among dynamics with social-safety-awareness: a reinforcement learning approach from 2d laser scans

J. Jin, N. M. Nguyen, N. Sakib, D. Graves, H. Yao, and M. Jagersand
2020 IEEE international conference on robotics and automation (ICRA)
Paper

A geometric perspective on visual imitation learning

J. Jin, L. Petrich, M. Dehghan, and M. Jagersand
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Paper

Visual geometric skill inference by watching human demonstration

J. Jin, L. Petrich, Z. Zhang, M. Dehghan, and M. Jagersand
2020 IEEE International Conference on Robotics and Automation (ICRA)
Paper

Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach

J. Jin, L. Petrich, Z. Zhang, M. Dehghan, and M. Jagersand
2019 International Conference on Robotics and Automation (ICRA)
Paper

Online object and task learning via human robot interaction

M. Dehghan, Z. Zhang, M. Siam, J. Jin, L. Petrich, and M. Jagersand
2019 IEEE International Conference on Robotics and Automation (ICRA)
Paper

Real-Time Edge Template Tracking via Homography Estimation

X. Qin, S. He, Z. Zhang, M. Dehghan, J. Jin, and M. Jagersand
2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Paper

Work with me

Thanks for your interest in working with me. I welcome students from all levels and international scholars to visit my lab. Please visit this page for more details.