Haofan Wang

Work Email: wanghaofan (at) xiaohongshu.com
School Email: haofanw (at) alumni.cmu.edu
Personal Email: haofanwang.ai (at) gmail.com
Research Interests
  • Consistent Image Generation
  • Temporally Consistent Video Generation
  • Fast Inference, Fine-grained Control, Multi-Features Disentanglement
Biography

Haofan Wang (王浩帆) is a research engineer at InstantX and Xiaohongshu, working primarily on controllable and conditional content generation (AIGC). Before that, he worked at MMU, Kuaishou. He obtained master degree from Carnegie Mellon University. He visited Image and Video Understanding Laboratory (IVUL), King Abdullah University of Science and Technology (KAUST) in 2020 and DATA Lab, Texas A&M University in 2019, repectively. He studied at UC Berkeley in 2017 as a visiting student, and worked as undergraduate research assistant at Berkeley RISE Lab, collaborating with Corey Zumar on Clipper. He also spent time at OpenMined, RealAI, Everspry, Horizon Robotics and Institute of Software Chinese Academy of Sciences. His highlighted works inlcude InstantStyle, InstantID, Score-CAM.

Selected Publications (Full List)
[motion]
InstantStyle : Free Lunch towards Style-Preserving in Text-to-Image Generation
Haofan Wang*, Qixun Wang, Xu Bai, Zekui Qing, and Anthony Chen. * means corresponding author.
Arxiv 2024
[Project Page] [Code]
[motion]
InstantID : Zero-shot Identity-Preserving Generation in Seconds
Qixun Wang, Xu Bai, Haofan Wang*, Zekui Qing, and Anthony Chen. * means corresponding author.
Arxiv 2024
[Project Page] [Code]
[motion]
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Anthony Chen, Huanrui Yang, Yulu Gan, Denis A Gudovskiy, Zhen Dong, Haofan Wang, Tomoyuki Okuno, Yohei Nakata, Shanghang Zhang, and Kurt Keutzer.
Arxiv 2023
[Paper] [Project Page]
[motion]
Multimodal Sense-Informed Forecasting of 3D Human Motions
Zhenyu Lou, Qiongjie Cui, Haofan Wang, Xu Tang, and Hong Zhou.
CVPR 2024
[Paper] [Project Page]
[motion]
Expressive Forecasting of 3D Whole-body Human Motions
Pengxiang Ding, Qiongjie Cui, Haofan Wang, Min Zhang Mengyuan Liu and Donglin Wang.
AAAI 2024
[Paper] [Project Page]
[motion]
Synthesizing Physically Plausible Human Motions in 3D Scenes
Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, and Yangang Wang.
3DV 2024
[Paper] [Code] [Project Page]
[motion]
Test-time Personalizable Forecasting of 3D Human Poses
Qiongjie Cui, Huaijiang Sun, Jianfeng Lu, Weiqing Li, Bin Li, and Haofan Wang.
ICCV 2023, Poster
[Paper] [Project Page]
[elict]
One-shot Implicit Animatable Avatars with Model-based Priors
Yangyi Huang*, Hongwei Yi*, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, and Cai Deng. * means equal contribution.
ICCV 2023, Poster
[Paper] [Project Page]
[tmm]
Show Me a Video: A Large-Scale Narrated Video Dataset for Coherent Story Illustration
Yu Lu, Feiyue Ni, Haofan Wang, Xiaofeng Guo, Linchao Zhu, Zongxin Yang, Ruihua Song, Lele Cheng, and Yi Yang.
IEEE Transactions on Multimedia (TMM), 2023
[Paper] [Project Page]
[kgrn]
Knowledge-aware Global Reasoning for Situation Recognition
Weijiang Yu*, Haofan Wang*, Guohao Li, Nong Xiao, and Bernard Ghanem. * means equal contribution.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2023
[Paper] [Code]
[smooth]
ECLIP: Efficient Contrastive Language-Image Pretraining via Ensemble Confidence Learning and Masked Language Modeling
Jue Wang*, Haofan Wang*, Weijia Wu, Jincan Deng, Yu Lu, Xiaofeng Guo, and Debing Zhang. * means equal contribution.
ICML 2022 Pre-training Workshop
[Paper] [Code] [Citation: 11]
[smooth]
Smoothed Geometry for Robust Attribution
Zifan Wang, Haofan Wang, Shakul Ramkumar, Matt Fredrikson, Piotr Mardziel, and Anupam Datta
NeurIPS 2020, Poster
[Paper] [Poster] [Code] [Citation: 29]
[Score-CAM]
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel and Xia (Ben) Hu
IEEE CVPR 2020 Workshop on Fair, Data Efficient and Trusted Computer Vision
[Project Page] [Paper] [Slides] [News] [Citation: 531]
Talks
  • [02/2024] MMLab@NTU & ShowLab@NUS & Guest Lecture@NUS CS5260. InstantID: Zero-shot Identity-Preserving Generation in Seconds
  • [12/2022] Xiaohongshu. A Review of Recent Progress in AIGC and its Applications
  • [06/2021] MMU, Kuaishou. An Overview of Large Scale Pre-Training
    • Alumni Interns
  • Yangyi Huang (MS@ZJU) -> Next: Research Intern@NVIDIA
  • Liang Pan (MS@SEU) -> Next: Research Intern@Shanghai AI Lab
  • Pengxiang Ding (MS@BUPT) -> Next: Ph.D@Westlake University
  • Yifan Yao (MS@SJTU) -> Next:China Life Insurance Company Limited
    • Teaching
  • TA of 18-793 Image and Video Processing, Prof: Aswin C Sankaranarayanan, Summer 2020, Carnegie Mellon University
    • Professional Services
  • Journal Reviewer: IJCV, TMM, TKDD, AI Communications
  • Conference Reviewer: NeurIPS (2021,2022,2023), ICLR (2022,2023,2024), ICML (2022,2023), ECCV (2022,2024), CVPR (2023,2024), XAI4CV@CVPR (2023), ICCV (2023), PRCV (2023）
    • Career Trajectory

    Kuaishou (2021-2022) --> Xiaohongshu (2022-) --> What's NeXT? --> Digital Nomad (203x)

    Misc

    Piano Level 7 (Yamaha YU118DN, Roland HP704), Scuba Diving (AOW), Snowboarding (elementatry), Amateur photographer (Sony RX100M5, Mavic Air, Instax Mini90, Osmo Pocket3), Janpanese (elementary), US Stock player