Short Bio
External Profiles
I am a tenure‑track Assistant Professor of Computer Science & Systems, jointly appointed in the UW Graduate School and the Tacoma School of Engineering & Technology at the University of Washington. I direct the Multi‑Foundation Model Lab (MF‑Lab), advise Ph.D. students in the Computer Science & Systems Ph.D. program, and since June 2025 have served as a Visiting Faculty Researcher with the Meta AI AGI team. My research on Multimodal Generative AI, Embodied and Robotic Intelligence, and Intelligent Transportation—in close collaboration with the Paul G. Allen School of Computer Science & Engineering and the Washington State Transportation Center (TRAC)—has appeared in CVPR, ICCV, NeurIPS, ICLR, AAAI, IJCAI and ACM MM, and contributed to The Washington Post investigations that earned the 2022 Pulitzer Prize for Public Service.
Prior to UW I spent seven years at Carnegie Mellon University’s Language Technologies Institute, rising from Research Associate and Post‑doc to Project Scientist / Instructor under the mentorship of Prof. Alexander G. Hauptmann and Prof. Teruko Mitamura, and collaborating closely with Prof. David R. Mortensen (KAIROS TA‑1 & Eratosthenes backend), and Prof. Alan W. Black (KAIROS TA‑2). I served as technical lead for DARPA KAIROS and KAIROS‑Plus Projects, and contributed to IARPA DIVA and NIST PSIAP Projects. Earlier, I completed research internships at Alibaba DAMO Academy, Google Brain, and Microsoft Research; and received the Intel Ph.D. Fellowship and IBM Outstanding Student Scholarship.
My research has been featured by The Washington Post, The New York Times, CBS News and other outlets, influencing public‑service reporting, safety analytics and mobility policy.
Interested in joining? I am recruiting Ph.D. students and postdoctoral researchers whose interests align with my research directions: Multimodal Generative AI, Embodied & Robotic Intelligence, and Intelligent Transportation. If you are excited about these areas, please email me or fill in this form.
News
- 25 Jun 2025 | One paper accepted at ICCV 2025.
- 15 Jun 2025 | Dr. Cheng will join the Meta AI AGI team as a Visiting Faculty Researcher.
- 12 Jun 2025 | Invited keynote at CVPR 2025 Anti‑UAV Workshop; our survey Securing the Skies: A Comprehensive Survey on Anti‑UAV Methods, Benchmarking, and Future Directions wins the workshop’s Best Paper Award.
- 31 Mar 2025 | Incoming Ph.D. student Yifei Dong receives UW GSFEI Top Scholar Award 2025.
- 28 Mar 2025 | Dr. Cheng serves as Area Chair for NAACL 2025.
- 10 Feb 2025 | Four papers accepted at CVPR 2025 (2 main‑conference, 2 workshop papers).
- 05 Feb 2025 | One paper accepted at ICLR 2025.
- 20 Jan 2025 | Two papers accepted at AAAI 2025, including one Oral presentation.
Research Interests
Multimodal Foundation Models
AI that reads, sees, and generates across modalities.
- Vision–language pre-training
- Instruction & alignment
- Emotion reasoning & affective computing
- Efficient adaptation (LoRA/PEFT)
Embodied / Robotic Intelligence
Agents that perceive, reason, and act safely.
- Language-guided navigation & manipulation
- Sim-to-real robustness
- Multimodal sensing & scene understanding
- Human-aware planning
Intelligent Transportation
Mobility as a living lab—measuring real-world impact.
- Traffic-video analytics
- Risk & driver-state modeling
- Mobility simulation
- Edge AI for real-time perception
Selected Publications
View my full publication list on Google Scholar ↗
2025 Emotion-LLaMA – NeurIPS 24 ▼
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
(NeurIPS 2024) Advances in Neural Information Processing Systems, 2024
Towards Calibrated Robust Fine-Tuning of Vision-Language Models
(NeurIPS 2024; CMU 11775 course project) Advances in Neural Information Processing Systems, 2024
Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions
(NeurIPS 2024 Spotlight) Advances in Neural Information Processing Systems, 2024
MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis
(ICLR 2025; WordArt; ~1M visits) International Conference on Learning Representations, 2025
Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios
(CVPR 2025) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2025
StableAnimator: High-Quality Identity-Preserving Human Image Animation
(CVPR 2025) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2025
Video-Grounded Dialogue: New dataset and metric for event-driven QA.
(AAAI 2025 Oral) AAAI Conference on Artificial Intelligence, 2025
POPoS: Improving Efficient and Robust Facial Landmark Detection with Parallel Optimal Position Search
(AAAI 2025) AAAI Conference on Artificial Intelligence, 2025
ProMQA: Question Answering Dataset for Multimodal Procedural Activity Understanding
(NAACL 2025 Oral) Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, 2025
DyRoNet: Dynamic Routing and Low-Rank Adapters for Autonomous-Driving Streaming Perception
(WACV 2025) IEEE/CVF Winter Conference on Applications of Computer Vision, 2025
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision–Language Models for Universal Cross-Domain Retrieval
(WACV 2025) IEEE/CVF Winter Conference on Applications of Computer Vision, 2025
Black-Box Visual Prompting for Robust Adaptation of Foundation Models
(BlackVIP (CVPR'23 ext.), submitted to TPAMI) IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
2024 BlockGCN - CVPR 24 ▼
BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition
(CVPR 2024) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2024
MotionEditor: Editing Video Motion via Content-Aware Diffusion
(CVPR 2024) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2024
PROS: Prompting-to-Simulate Generalized Knowledge for Universal Cross-Domain Retrieval
(CVPR 2024) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2024
Shield: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply-Chain Disruptions
(EMNLP 2024 Oral, Industry Track) Conference on Empirical Methods in Natural Language Processing, 2024
DCPT: Darkness Clue-Prompted Tracking in Night-Time UAVs
(ICRA 2024) IEEE International Conference on Robotics and Automation, 2024
FaceChain-ImagineID: High-Fidelity Diverse Talking Faces from Disentangled Audio
(CVPR 2024; Part of FaceChain (9.3K GitHub stars)) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2024
WordArt Designer API: Artistic Typography with LLMs on ModelScope
Spotlight @ NeurIPS ML4Creativity workshop
2023 DAMO-StreamNet - IJCAI 23 ▼
DAMO-StreamNet: Optimizing Streaming Perception in Autonomous Driving
(IJCAI 2023) International Joint Conference on Artificial Intelligence, 2023
HDFormer: High-Order Directed Transformer for 3D Human Pose Estimation
(IJCAI 2023) International Joint Conference on Artificial Intelligence, 2023
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
(ICCV 2023 Oral) International Conference on Computer Vision, 2023
ChartReader: A Unified Framework for Chart Derendering and Comprehension Without Heuristic Rules
(ICCV 2023) International Conference on Computer Vision, 2023
WordArt Designer: User-Driven Artistic Typography Synthesis Using Large Language Models
(EMNLP 2023, Industry Track) Conference on Empirical Methods in Natural Language Processing, 2023
KeyPosS: Plug-and-Play Facial Landmark Detection through GPS-Inspired True-Range Multilateration
(ACM-MM 2023) ACM International Conference on Multimedia, 2023
Posynda: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation
(ACM-MM 2023) ACM International Conference on Multimedia, 2023
≤ 2022 Rethinking Spatial Invariance – CVPR 22 ▼
Rethinking Spatial Invariance of Convolutional Networks for Object Counting
(CVPR 2022; used for WaPo's Pulitzer coverage) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2022
GSRFormer: Grounded Situation Recognition Transformer with Alternate Semantic Attention
(ACM-MM 2022 Oral) ACM International Conference on Multimedia, 2022
Learning Spatial Awareness to Improve Crowd Counting
(ICCV 2019 (Oral); CMU's INF-Public-Safety-Tools) International Conference on Computer Vision, 2019
Video2Shop: Exact Matching Clothes in Videos to Online Shopping Images
(CVPR 2017; Alibaba Pailitao (~500M users)) IEEE/CVF Computer Vision and Pattern Recognition Conference, 2017
Video eCommerce++: Toward Large-Scale Online Video Advertising
(ACM-MM 2017 Oral; ACM-SCF Best Student Paper) ACM International Conference on Multimedia, 2016