Rain Tianyu Sun

Ph.D. student, College of Computing and Data Science, Nanyang Technological University.

I am a Ph.D. student in computer science at Nanyang Technological University, advised by Prof. Guosheng Lin. My research interests include video generation, 3D computer vision, and robot learning. I am currently working on motion representations for character animation.

I received my M.E. from the Department of Electronic Engineering at Tsinghua University in 2024, fortunately advised by Prof. Guijin Wang. Before that, I obtained my B.E. in EE and B.Ec in SEM in 2021. I also had the pleasure of collaborating with Prof. Paul Bogdan (USC), Prof. Jing-hao Xue (UCL), and Prof. Hengshuang Zhao (HKU).

I'm interested in human-centric video generation and robot learning. My current work focuses on generating realistic human videos in complex, interactive environments, especially those involving human–object interactions. More broadly, I’m interested in how humans engage with the physical world, and in the long term, I aim to develop models that can understand and simulate rich human–object manipulation behaviors.

News

[2025.2] One paper on multi-view human video generation has been released.

[2025.5] One paper on human video generation for Robot2Human handover has been released.

Research

	RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction Tianyu Sun, Zhoujie Fu, Zihui Gao, Bang Zhang, Guosheng Lin Pre-Print Project \| Paper \| Code Formulated a sim-to-real perception pipeline leveraging generative video priors and morphology-based depth editing to scale multimodal handover datasets (Hand2Bot) and achieve zero-shot intention gating on physical robotic platforms.
	Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning Tianyu Sun, Jianhao Li, Xueqian Zhang, Zhongdao Wang, Bailan Feng, Ke Xu, Hengshuang Zhao IEEE International Conference on Robotics and Automation (ICRA), 2025 Paper A simple yet effective Gumbel Spatial Pruning (GSP) layer that dynamically prunes points based on a learned end-to-end sampling.
	Variation-Robust Few-Shot 3D Affordance Segmentation for Robotic Manipulation Dingchang Hu, Tianyu Sun*, Pengwei Xie, Siang Chen, Huazhong Yang, Guijin Wang IEEE Robotics and Automation Letters (RA-L)*, 2025 Paper An orientation-tolerant feature extractor and multi-scale label propagation for robust few-shot affordance segmentation.
	Segment, Lift and Fit: Automatic 3D Shape Labeling from 2D Prompts Jianhao Li, Tianyu Sun*, Zhongdao Wang, Enze Xie, Bailan Feng, Hongbo Zhang, Ze Yuan, Ke Xu, Jiaheng Liu, Ping Luo European Conference on Computer Vision (ECCV)*, 2024 Paper A Segment–Lift–Fit paradigm that predicts 3D shapes of objects from 2D priors without requiring dataset-specific training.
	Part-guided 3D RL for Sim2Real articulated object manipulation Pengwei Xie, Rui Chen, Siang Chen, Yuzhe Qin, Fanbo Xiang, Tianyu Sun, Jing Xu, Guijin Wang, Hao Su IEEE Robotics and Automation Letters (RA-L)*, 2023 Paper \| Code \| Video A part-guided 3D reinforcement learning framework for articulated object manipulation without demonstrations.
	Diffusion-based depth inpainting for transparent and reflective objects Tianyu Sun, Dingchang Hu, Yixiang Dai, Guijin Wang IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024 Paper A diffusion-based depth inpainting framework for transparent and reflective surfaces.
	TROSD: A New RGB-D Dataset for Transparent and Reflective Object Segmentation in Practice Tianyu Sun, Guodong Zhang, Wenming Yang, Jing-hao Xue, Guijin Wang IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023 Project \| Paper \| Github \| Code Ocean \| Data A large-scale RGB-D dataset and model for transparent and reflective object segmentation.

Professional Services

Conference Reviewer: ICCV, ECCV, ICRA, IROS.

Journal Reviewer: TCSVT, RA-L, KBS.

Teaching

Teaching Assistant, Pattern Recognition (60230023-0), Spring 2023

Teaching Assistant, Digital Image Processing (30230703-1), Spring 2024

This homepage is heavily built on the source code of homepage of Jon Barron.