Haofan Wang

Work Email: wanghaofan (at) xiaohongshu.com
School Email: haofanw (at) alumni.cmu.edu
School Email: haofanw (at) andrew.cmu.edu
School Email: haofanwang (at) berkeley.edu
Personal Email: haofanwang.ai (at) gmail.com
Research Interests
  • Controllable Image/Video/3D Generation
  • Designing Simple, Pluggable and Effective Modules
  • Trustworthy Machine Learning and Foundation Models
Biography

Haofan Wang (王浩帆) is a research engineer at Xiaohongshu, working primarily on controllable and conditional content generation (AIGC). He regularlly contributes to open source communities, including diffusers and ColossalAI. Before that, he worked at MMU, Kuaishou. He obtained master degree from Carnegie Mellon University. He visited Image and Video Understanding Laboratory (IVUL), King Abdullah University of Science and Technology (KAUST) in 2020 and DATA Lab, Texas A&M University in 2019, repectively. He studied at UC Berkeley from August 2017 to December 2017 as a visiting student, and worked as undergraduate research assistant at Berkeley RISE Lab, collaborating with Corey Zumar on Clipper. He also spent time at OpenMined, RealAI, Everspry, Horizon Robotics and Institute of Software Chinese Academy of Sciences. Some of the earlier works he co-authored are Score-CAM.

  • [Update:09/04/2023]. I have multiple opening internship positions for controllable image/video generation, please email me if you are interested in! For research interns, the minimum requirement is to have published at least one paper at top conferences (CVPR/ICCV/ECCV/etc).
    • Selected Publications (Full List)
    [motion]
    Synthesizing Physically Plausible Human Motions in 3D Scenes
    Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, Yangang Wang
    Arxiv 2023
    [Paper] [Code] [Project Page]
    [motion]
    Test-time Personalizable Forecasting of 3D Human Poses
    Qiongjie Cui, Huaijiang Sun, Jianfeng Lu, Weiqing Li, Bin Li, and Haofan Wang,
    ICCV 2023, Poster
    [Paper] [Project Page]
    [elict]
    One-shot Implicit Animatable Avatars with Model-based Priors
    Yangyi Huang*, Hongwei Yi*, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, and Cai Deng. * means equal contribution.
    ICCV 2023, Poster
    [Paper] [Project Page]
    [tmm]
    Show Me a Video: A Large-Scale Narrated Video Dataset for Coherent Story Illustration
    Yu Lu, Feiyue Ni, Haofan Wang, Xiaofeng Guo, Linchao Zhu, Zongxin Yang, Ruihua Song, Lele Cheng, and Yi Yang
    IEEE Transactions on Multimedia (TMM), 2023
    [Paper] [Project Page]
    [kgrn]
    Knowledge-aware Global Reasoning for Situation Recognition
    Weijiang Yu*, Haofan Wang*, Guohao Li, Nong Xiao, and Bernard Ghanem. * means equal contribution.
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2023
    [Paper] [Code]
    [smooth]
    ECLIP: Efficient Contrastive Language-Image Pretraining via Ensemble Confidence Learning and Masked Language Modeling
    Jue Wang*, Haofan Wang*, Weijia Wu, Jincan Deng, Yu Lu, Xiaofeng Guo, and Debing Zhang. * means equal contribution.
    ICML 2022 Pre-training Workshop
    [Paper] [Code] [Citation: 11]
    [smooth]
    Smoothed Geometry for Robust Attribution
    Zifan Wang, Haofan Wang, Shakul Ramkumar, Matt Fredrikson, Piotr Mardziel, and Anupam Datta
    NeurIPS 2020, Poster
    [Paper] [Poster] [Code] [Citation: 29]
    [Score-CAM]
    Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
    Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel and Xia (Ben) Hu
    IEEE CVPR 2020 Workshop on Fair, Data Efficient and Trusted Computer Vision
    [Project Page] [Paper] [Slides] [News] [Citation: 531]
    [HeadPose]
    Hybrid Coarse-fine Classification for Head Pose Estimation
    Haofan Wang, Zhenghua Chen, and Yi Zhou
    ArXiv 1901.06778
    [Project Page] [Paper] [Citation: 12]
    Mentoring
  • Liang Pan (MS@SEU)
  • Anthony Chen (MS@PKU)
  • Shangkun Sun (Ph.D@PKU)
    • Alumni Interns
  • Pengxiang Ding (MS@BUPT) -> Next: Ph.D@Westlake University
  • Yifan Yao (MS@SJTU) -> Next:China Life Insurance Company Limited
    • Teaching
  • TA of 18-793 Image and Video Processing, Prof: Aswin C Sankaranarayanan, Summer 2020, Carnegie Mellon University
    • Professional Services
  • Journal Reviewer: IJCV, TMM, AI Communications
  • Conference Reviewer: NeurIPS (2021,2022,2023), ICLR (2022,2023,2024), ICML (2022,2023), ECCV (2022), CVPR (2023), XAI4CV@CVPR (2023), ICCV (2023), PRCV（2023）
    • Career Trajectory

    Kuaishou (2021-2022) --> Xiaohongshu (2022-) --> What's NeXT?

    Misc

    Piano Level 7 (Yamaha YU118DN, Roland HP704), Amateur photographer (Sony RX100M5, Instax mini90), Janpanese novice, US Stock player