About Me

Sheng Jin is currently a Phd candidate (2020-present) at the University of Hong Kong (HKU), advised by Dr. Ping Luo and co-supervised by Prof. Wenping Wang and Prof. Xiaoou Tang. He is also a part-time researcher (2020-present) at SenseTime Research mentored by Dr. Wentao Liu.

In 2020, he received his master's degree in the Department of Automation at Tsinghua University, advised by Prof. Changshui Zhang. In 2017, he received the B.Eng. degree with highest honor (Outstanding Graduate Scholarships) from Tsinghua University.

His research focus is on teaching machines/robots to see and understand human behaviors such as human body poses, actions, and human-machine interactions.

News


  • We are recruiting intern researchers in computer vision at SenseTime Research (Beijing). If you are interested in, please send your CV to my email.

  • [2021-07] One paper accepted by ICCV'2021.

  • [2021-03] Two papers accepted by CVPR'2021.

  • [2020-09] One paper accepted by NeurIPS'2020.

  • [2020-08] We have released MMPose Toolbox v0.6.

  • [2020-07] Two papers accepted by ECCV'2020.

  • [2020-06] Our team won the 3rd place of ACM MM'2020 HiEve Challenge (Leaderboard: SimpleTrack).
  • [2020-04] One paper accepted by TNNLS.

  • [2020-02] One paper accepted by AAAI'2020 (Oral).

Education

The University of Hong Kong

PhD in Computer Science (HKPFS awardee), 2020~now
Tsinghua University

MS in Control Science and Engineering, 2017~2020
Tsinghua University

BSc in Automation (ranking 1/145), 2013~2017


Honors and Awards

  • YS and Christabel Lung Postgraduate Scholarship, 2020-2021.

  • HKU Presidential PhD Scholarship (HKU-PS) 2020.

  • Hong Kong PhD Fellowships (HKPF), 2020.

  • Outstanding Graduate Scholarship, Tsinghua University (top 1% in Tsinghua), 2017.

  • The Baosteel Excellent Student Scholarship, 2016.

  • Zheng Weimin Scholarship (2nd class) for Comprehensive Excellence, 2016.

  • Tsinghua-JJWorld (Beijing) Nework Technology Fellowships, Tsinghua University, 2015.

  • Tsinghua-Evergrande Fellowships for Academic Excellence, Tsinghua University, 2014.


Selected Publications

Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images

Size Wu, Sheng Jin, Wentao Liu, Lei Bai, Chen Qian, Dong Liu, Wanli Ouyang

IEEE International Conference on Computer Vision (ICCV), 2021.

[Paper
ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search

Lumin Xu, Yingda Guan, Sheng Jin, Wentao Liu, Chen Qian, Ping Luo, Wanli Ouyang, Xiaogang Wang

Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

[Paper]  [Code]
When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks

Jiahang Wang, Sheng Jin, Wentao Liu, Weizhong Liu, Chen Qian, Ping Luo

Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

[Paper]  [Code]
When Counterpoint Meets Chinese Folk Melodies

Nan Jiang, Sheng Jin, Zhiyao Duan, Changshui Zhang

Conference on Neural Information Processing Systems (NeurIPS), 2020.

[Paper] [Supplementary] [Poster] [Code] [Project Page]
Whole-Body Human Pose Estimation in the Wild

Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, and Ping Luo

European Conference on Computer Vision (ECCV), 2020.

[Paper] [Dataset]
Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation

Sheng Jin, Wentao Liu, Enze Xie, Wenhai Wang, Chen Qian, Wanli Ouyang, and Ping Luo

European Conference on Computer Vision (ECCV), 2020.

[Paper] [Blog(知乎)]
TRB: A Novel Triplet Representation for Understanding 2D Human Body

Haodong Duan, Kwan-Yee Lin, Sheng Jin, Wentao Liu, Chen Qian, Wanli Ouyang

IEEE International Conference on Computer Vision (ICCV), 2019, Oral.

[Paper] [Dataset]
Robust Few-Shot Learning for User-Provided Data

Jiang Lu, Sheng Jin, Jian Liang, and Changshui Zhang

IEEE Transactions on Neural Networks and Learning Systems (TNNLS).

[Paper
RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning

Nan Jiang, Sheng Jin, Zhiyao Duan, Changshui Zhang

AAAI Conference on Artificial Intelligence (AAAI), 2020, Oral.

[Paper] [Demo]
Multi-person Articulated Tracking with Spatial and Temporal Embeddings

Sheng Jin, Wentao Liu, Wanli Ouyang, Chen Qian

Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

[Paper] [Demo]
Hierarchical Automatic Curriculum Learning: Converting a Sparse Reward Navigation Task into Dense Reward

Nan Jiang, Sheng Jin, Changshui Zhang

Neurocomputing, 2019.

[Paper
Connectionist Temporal Classification with Maximum Entropy Regularization

Hu Liu, Sheng Jin, Changshui Zhang

Conference on Neural Information Processing Systems (NeurIPS), 2018, Spotlight.

[Paper] [Poster] [Code]
Towards Multi-Person Pose Tracking: Bottom-up and Top-down Methods

Sheng Jin, Xujie Ma, Zhipeng Han, Yue Wu, Wei Yang, Wentao Liu, Chen Qian, Wanli Ouyang

International Conference on Computer Vision (ICCV) PoseTrack Workshop, 2017.

[Paper] [Leaderboard](BUTDS and BUTD2) [Demo]


Projects

MMPose Toolbox

MMPose is an open-source toolbox for pose estimation based on PyTorch, which is a part of the OpenMMLab project.

[Project
ACM MM'2020 HiEve Challenge

Our team (SimpleTrack) won the 3rd place in Track-3 "Crowd Pose Tracking in Complex Events" of ACM MM'2020 HiEve Challenge.

[Leaderboard] [Technical Report]
CVPR'2018 Look Into Person (LIP) Challenge

Our team (MJDG) won the 2nd place in Track-4 "Multi-Human Pose Estimation Challenge" of CVPR'2018 LIP Challenge.

[Leaderboard] [Oral Presentation]  
ICCV'2017 PoseTrack Challenge

Our team (BUTDS | BUTD2) won the 2nd places in both Track-1 "Single-Frame Person Pose Estimation" and Track-3 "Multi-Person Pose Tracking" of ICCV'2017 PoseTrack Challenge.

[Leaderboard] [Technical Report]  [Oral Presentation]  [Demo]


Patents

Click to expand or collapse
Key point detection method, device, electronic equipment and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN111898642A. Publication Date: 2020-11-06.
Key point detection method, device, electronic equipment and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN111783882A. Publication Date: 2020-10-16.
Image processing method and device, detection device and storage medium

Tong Li, Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN111539992A. Publication Date: 2020-08-14.
Key point detection method, device, electronic equipment and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN111444928A. Publication Date: 2020-07-24.
Image processing method and device, detection device and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN109948526A. Publication Date: 2019-06-28.
Image processing method and device, detection device and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN109934183A. Publication Date: 2019-06-25.
Deep learning model training method and device, training equipment and storage medium

Sheng Jin, Wentao Liu, Chen Qian

Chinese Invention Patent.

Publication Number: CN109919245A. Publication Date: 2019-06-21.


Activities


  • Conference Reviewer/PC Member
    • NeurIPS'19, AAAI'19, NeurIPS'20, ICML'20, CVPR'20, AAAI'21, WACV'21, ICLR'21, ICML'21, CVPR'21, ICCV'21
  • Journal Reviewer
    • IEEE Transactions on Artificial Intelligence (TAI), Transactions on Image Processing (TIP), International Journal of Computer Vision (IJCV)
  • Website Chairs


  • Contacts

    js20 [at] connect.hku.hk | jinsheng [at] sensetime.com