Hi there! My name is Zhengyi Luo (罗正宜) and I am a final year PhD student at Carnegie Mellon University’s Robotics Institute, School of Computer Science, advised by Prof. Kris Kitani. I earned my bachelor’s degree from University of Pennsylvania in 2019, where I worked with Prof. Kostas Daniilidis. My research has been supported by Qualcomm Innovation Fellowship and the Meta AI Mentorship Program.
My research interest lies at the intersection of vision, learning, and robotics. I am working on topcis including human pose estimation, human-object interaction, human motion modelling etc. Through my research, I want to create methods that effectively interpret spatial-temporal sensory input and build a representation of the 3D world to reason about the interactions between agents and the physical environment. On the application side, I am excited about humanoid robots and AR/VR.
Publications
- HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
-
Tairan He*, Wenli Xiao*, Toru Lin,
Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu
- In submission
-
[Paper]
- CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
-
Guy Tevet, Sigal Raab, Setareh Cohan, Daniele Reda,
Zhengyi Luo, Xue Bin Peng, Amit H. Bermano, Michiel van de Panne
- In submission
-
[Paper]
[Project]
[Code]
- Humanoidlympics: Sports Environments for Physically Simulated Humanoids
-
Zhengyi Luo*, Jiashun Wang*, Kangni Liu*, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, Jessica Hodgins, Kris Kitani
- In submission
- * Equal contribution
-
[Paper]
[Project]
[Code]
- SkillMimic: Learning Reusable Basketball Skills from Demonstrations
-
Yinhuai Wang*, Qihan Zhao*, Runyi Yu*, Ailing Zeng, Jing Lin,
Zhengyi Luo, Hok Wai Tsui, Jiwen Yu, Xiu Li, Qifeng Chen, Jian Zhang, Lei Zhang, Ping Tan
- In submission
-
[Paper]
[Project]
[Code]
- Grasping Diverse Objects with Simulated Humanoids
-
Zhengyi Luo*, Jinkun Cao*, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu
- NeurIPS 2024
- * Equal contribution
-
[Paper]
[Project]
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
-
Tairan He*,
Zhengyi Luo*, Xialin He*, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, Guanya Shi
- CORL 2024
- * Equal contribution
-
[Paper]
[Project]
- Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
-
Tairan He*,
Zhengyi Luo*, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, Guanya Shi
- IROS 2024 (Oral Presentation)
- * Equal contribution
-
[Paper]
[Project]
- Real-Time Simulated Avatar from Head-Mounted Sensors
-
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu
- CVPR 2024 (Highlight — Top 11%)
-
[Paper]
[Project]
[Code]
- PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
-
Jingbo Wang*,
Zhengyi Luo*, Ye Yuan, Yixuan Li, Bo Dai
- CVPR 2024
- * Equal contribution
-
[Paper]
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
-
K Grauman et al.
- CVPR 2024 (Oral — Top 3%)
-
[Paper]
[Project]
- PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction
-
Yinhuai Wang, Jing Lin, Ailing Zeng,
Zhengyi Luo, Jian Zhang, Lei Zhang
- In Submission
-
[Paper]
[Project]
- Universal Humanoid Motion Representations for Physics-Based Control
-
Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu
- ICLR 2024 (Spotlight — Top 5%)
-
[Paper]
[Project]
[Code]
- Perpetual Humanoid Control for Real-time Simulated Avatars
-
Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu
- ICCV 2023
-
[Video]
[Paper]
[Project]
[Code]
- Learning Human Dynamics in Autonomous Driving Scenarios
-
Jingbo Wang, Ye Yuan,
Zhengyi Luo, Kevin Xie, Dahua Lin, Umar Iqbal, Sanja Fidler, Sameh Khamis
- ICCV 2023
-
[Paper]
- Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
-
Davis Rempe*,
Zhengyi Luo*, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, Or Litany
- CVPR 2023
- * Equal contribution
-
[Video]
[Paper]
[Project]
- Embodied Scene-aware Human Pose Estimation
-
Zhengyi Luo*, Shun Iwase*, Ye Yuan, Kris M. Kitani
- NeurIPS 2022
- * Equal contribution
-
[Video]
[Paper]
[Project]
[Code]
- From Universal Humanoid Control to Automatic Physically Valid Character Creation
-
Zhengyi Luo, Ye Yuan, Kris M. Kitani
- ArXiv
-
[Paper]
[Project]
[Code]
- Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
-
Ye Yuan, Yuda Song,
Zhengyi Luo, Wen Sun, Kris M. Kitani
- ICLR 2022 (Oral Presentation — Top 1.6%)
-
[Paper]
[Project]
[Code]
- Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
-
Zhengyi Luo, Ryo hachiuma, Ye Yuan, Kris M. Kitani
- ICCVW (CV4ARVR) 2021, NeurIPS 2021
-
[Video]
[Paper]
[Project]
[Code]
- 3D Human Motion Estimation via Motion Compression and Refinement
-
Zhengyi Luo, S. Alireza Golestaneh, Kris M. Kitani
- ACCV 2020
- Oral Presentation
-
[Video]
[Paper]
[Project]
[Code]
- Learning Shape Representations for Clothing Variations in Person Re-Identification
-
Yu-Jhe Li,
Zhengyi Luo, Xinshuo Weng, Kris M. Kitani
- arXiv, 2020
-
[Paper]
- Cross-Domain 3D Equivariant Image Embeddings
-
Carlos Esteves, Avneesh Sud,
Zhengyi Luo, Kostas Daniilidis, Ameesh Makadia
- ICML 2019
-
[Paper]
- Cloud Chaser: Real Time Deep Learning Computer Vision on Low Computing Power Devices
-
Zhengyi Luo, Austin Small, Liam Dugan, Stephen Lane
- ICMV 2018
-
[Paper]
[Project]
[Code]
- The rural–urban stress divide: Obtaining geographical insights through Twitter
-
Kokil Jaidka, Sharath Chandra Guntuku, Jane H Lee,
Zhengyi Luo, Anneke Buffone, Lyle H Ungar
- Computers in Human Behavior, 2020: 106544
-
[Paper]
[Project]
---
Professional Services
Conference Reviewer: ICML (2020, 2021, 2022, 2023), ICLR (2022, 2023), NeurIPS (2021, 2022), CVPR (2022, 2023), ICCV (2021), ECCV (2022)
Journal Reviewer: IJCV, TMM
