Robot Learning Seminar: Shao-Hua Sun (NTU) and Edward S. Hu (UPenn) on "Program Guided Robot Learning" and "The Sensory Needs of Robot Learners".

Please join us tomorrow at 4:00 PM for guest lectures for the Robot Learning class. Shao-Hua Sun (NTU) and Edward S. Hu (UPenn) will share their recent work on robot learning. Details below. We look forward to seeing you there!

Location: Engineering Hall 1, A542

Time: March 22, 2024 (Fri) 16:00 ~ 17:30

Speaker #1: Shao-Hua Sun (Assistant Professor at NTU)

Title: Program Guided Robot Learning

Abstract: Recent development in artificial intelligence and machine learning has remarkably advanced machines’ ability to understand images and videos, comprehend natural languages and speech, and outperform human experts in complex games. However, building intelligent robots that can operate in unstructured environments, manipulate unknown objects, and acquire novel skills – to free humans from tedious or dangerous manual work – remains challenging. My research focuses on developing a robot learning framework that enables robots to acquire long-horizon and complex skills with hierarchical structures, such as furniture assembly and cooking. Specifically, I present an interpretable and generalizable program-guided robot learning framework, which represents desired behaviors as a program and acquires primitive skills for executing desired skills. This talk will discuss a series of projects toward building this framework.

Bio: Shao-Hua Sun is an Assistant Professor at National Taiwan University (NTU) with a joint appointment in the Department of Electrical Engineering and the Graduate Institute of Communication Engineering. Prior to joining NTU, he completed his Ph.D. in Computer Science at the University of Southern California, where he worked in the Cognitive Learning for Vision and Robotics Lab (CLVR). Before that, he received his B.S. degree in Electrical Engineering from NTU. His research interests span Robot Learning, Reinforcement Learning, Program Synthesis, and Machine Learning.

Speaker #2: Edward S. Hu (PhD student at UPenn)

Title: The Sensory Needs of Robot Learners

Abstract: What kinds of environmental information does a robot need to efficiently learn new behaviors? We have a clue from our human experience; we look at our shoelaces as we first learn to tie them but having mastered this skill, can do it from touch alone. We call this phenomenon “sensory scaffolding”: observation streams that are not needed by a master might yet aid a novice learner. In this talk, I will present our recent efforts in investigating "sensory scaffolding" for robot learning - in the form of an algorithm that effectively exploits privileged observations, Scaffolder, as well as a variety of interesting experimental results that reveal what sensory requirements are induced by robot learning.

Bio: Edward S. Hu is a CS Ph.D. student at the University of Pennsylvania advised by Dinesh Jayaraman. He is broadly interested in artificial intelligence, ranging from virtual agents to physical robots. As a result, his research spans perception, reinforcement learning, and robotics. Edward received his BS/MS in CS from the University of Southern California, where he worked with Joseph J. Lim. This summer, He will be interning at Microsoft Research NYC, hosted by John Langford and Alex Lamb.