Haofan Wang

Personal Email: haofanwang.ai (at) gmail.com
Research Interests
  • Image Foundation Model
  • Consistent and Personalized Generation
  • Building exciting, shippable AI Products and Decentralized Platform
Biography

Haofan Wang (王浩帆) is a researcher at InstantX, working primarily on controllable and conditional content generation (AIGC). Before that, he worked at Xiaohongshu, and MMU (Known as Kolors Team), Kuaishou. He obtained master degree from Carnegie Mellon University. He visited Image and Video Understanding Laboratory (IVUL), King Abdullah University of Science and Technology (KAUST) in 2020 and DATA Lab, Texas A&M University in 2019, repectively. He studied at UC Berkeley in 2017 as a visiting student, and worked as undergraduate research assistant at Berkeley RISE Lab, collaborating with Corey Zumar on Clipper. He also spent time at OpenMined, RealAI, Everspry, Horizon Robotics and Institute of Software Chinese Academy of Sciences. His highlighted works inlcude InstantStyle, InstantID, Score-CAM. He accepts open-source sponsors via Ko-fi.

Selected Publications (Full List)
[motion]
InstantStyle-Plus :Style Transfer with Content-Preserving in Text-to-Image Generation
Haofan Wang*, Peng Xing, Renyuan Huang, Hao Ai, Qixun Wang, and Xu Bai. * means corresponding author.
Arxiv 2024
[Project Page] [Code]
[motion]
InstantStyle : Free Lunch towards Style-Preserving in Text-to-Image Generation
Haofan Wang*, Qixun Wang, Xu Bai, Zekui Qing, and Anthony Chen. * means corresponding author.
Arxiv 2024
[Project Page] [Code]
[motion]
InstantID : Zero-shot Identity-Preserving Generation in Seconds
Qixun Wang, Xu Bai, Haofan Wang*, Zekui Qing, and Anthony Chen. * means corresponding author.
Arxiv 2024
[Project Page] [Code]
[motion]
Unified Video-Language Pre-training with Synchronized Audio
Shentong Mo, Haofan Wang, Huaxia Li, and Xu Tang
Arxiv 2024
[Paper]
[motion]
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Anthony Chen, Huanrui Yang, Yulu Gan, Denis A Gudovskiy, Zhen Dong, Haofan Wang, Tomoyuki Okuno, Yohei Nakata, Shanghang Zhang, and Kurt Keutzer.
ICML 2024
[Paper] [Project Page]
[motion]
Multimodal Sense-Informed Forecasting of 3D Human Motions
Zhenyu Lou, Qiongjie Cui, Haofan Wang, Xu Tang, and Hong Zhou.
CVPR 2024
[Paper] [Project Page]
[motion]
Expressive Forecasting of 3D Whole-body Human Motions
Pengxiang Ding, Qiongjie Cui, Haofan Wang, Min Zhang Mengyuan Liu and Donglin Wang.
AAAI 2024
[Paper] [Project Page]
[motion]
Synthesizing Physically Plausible Human Motions in 3D Scenes
Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, and Yangang Wang.
3DV 2024
[Paper] [Code] [Project Page]
[motion]
Test-time Personalizable Forecasting of 3D Human Poses
Qiongjie Cui, Huaijiang Sun, Jianfeng Lu, Weiqing Li, Bin Li, and Haofan Wang.
ICCV 2023, Poster
[Paper] [Project Page]
[elict]
One-shot Implicit Animatable Avatars with Model-based Priors
Yangyi Huang*, Hongwei Yi*, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, and Cai Deng. * means equal contribution.
ICCV 2023, Poster
[Paper] [Project Page]
[tmm]
Show Me a Video: A Large-Scale Narrated Video Dataset for Coherent Story Illustration
Yu Lu, Feiyue Ni, Haofan Wang, Xiaofeng Guo, Linchao Zhu, Zongxin Yang, Ruihua Song, Lele Cheng, and Yi Yang.
IEEE Transactions on Multimedia (TMM), 2023
[Paper] [Project Page]
[kgrn]
Knowledge-aware Global Reasoning for Situation Recognition
Weijiang Yu*, Haofan Wang*, Guohao Li, Nong Xiao, and Bernard Ghanem. * means equal contribution.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2023
[Paper] [Code]
[smooth]
ECLIP: Efficient Contrastive Language-Image Pretraining via Ensemble Confidence Learning and Masked Language Modeling
Jue Wang*, Haofan Wang*, Weijia Wu, Jincan Deng, Yu Lu, Xiaofeng Guo, and Debing Zhang. * means equal contribution.
ICML 2022 Pre-training Workshop
[Paper] [Code] [Citation: 11]
[smooth]
Smoothed Geometry for Robust Attribution
Zifan Wang, Haofan Wang, Shakul Ramkumar, Matt Fredrikson, Piotr Mardziel, and Anupam Datta
NeurIPS 2020, Poster
[Paper] [Poster] [Code] [Citation: 29]
[Score-CAM]
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel and Xia (Ben) Hu
IEEE CVPR 2020 Workshop on Fair, Data Efficient and Trusted Computer Vision
[Project Page] [Paper] [Slides] [News] [Citation: 531]
Talks
  • [06/2024] A2M. [Info]
  • [03/2024] OpenMMLab & REDTech & Extreme MART. [Video]
  • [02/2024] MMLab@NTU & ShowLab@NUS. InstantID: Zero-shot Identity-Preserving Generation in Seconds, [Video]
  • [12/2022] Xiaohongshu. A Review of Recent Progress in AIGC and its Applications
  • [06/2021] MMU, Kuaishou. An Overview of Large Scale Pre-Training
    • Teaching
  • Guest Lecture of CS5260: Neural Networks and Deep Learning II, Prof: Yang You, Spring 2024, National University of Singapore
  • TA of 18-793 Image and Video Processing, Prof: Aswin C Sankaranarayanan, Summer 2020, Carnegie Mellon University
    • Professional Services
  • Journal Reviewer: IJCV, TMM, TKDD, AI Communications
  • Conference Reviewer: NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, XAI4CV@CVPR, PRCV
    • Career Trajectory

    Kuaishou (2021-2022) --> Xiaohongshu (2022-2024) --> Stealth Startup (2024-) --> What's NeXT?

    Misc

    Piano Level 7 (Roland HP704), Scuba Diving (AOW), Snowboarding (elementatry), Amateur photographer (Sony RX100M5, Mavic Air, Instax Mini90, Osmo Pocket3), Janpanese (elementary), US Stock player