A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques
Lownish Rai Sookha1 Nikhil Pakhale1 Mudasir Ganaie1 Abhinav Dhall2
1 Indian Institute of Technology Ropar, India
2 Monash University, Australia
[Paper]
Overview of Generic Motion Generation Pipeline of existing SOTAs. Given the input from the respective modalities, the methods generate desired body or face motion using appropriate representation techniques.

Abstract

Body and face motion play an integral role in communication. They convey crucial information on the participants. Advances in generative modeling and multi-modal learning have enabled motion generation from signals such as speech, conversational context and visual cues. However, generating expressive and coherent face and body dynamics remains challenging due to the complex interplay of verbal / non-verbal cues and individual personality traits. This survey reviews body and face motion generation, covering core concepts, representations techniques, generative approaches, datasets and evaluation metrics. We highlight future directions to enhance the realism, coherence and expressiveness of avatars in dyadic settings. To the best of our knowledge, this work is the first comprehensive review to cover both body and face motion.


Representation Techniques

(Top) Given the body image, the body pose and geometry are reconstructed using body representation techniques. Source: Pavlakos et al. (2019)
(Bottom) Given face images, the face is parameterized and reconstructed using 3DMM frameworks, effectively capturing the pose and expression. Source: Retsinas et al. (2024)

Motion Visualizations

Body motion generated using SMPL
Face reconstruction using FaceVerse
Talking Face Generating using EdTalk (Diffusion-based model)

Datasets

Dataset Category Dataset Year Region Paper Dataset
Text-Conditioned Motion Datasets MotionFix 2024 Body PDF Icon Dataset Icon
Motion-X 2023 Body PDF Icon Dataset Icon
CelebV-Text 2023 Face PDF Icon Dataset Icon
HumanML3D 2022 Body PDF Icon Dataset Icon
HUMANISE 2022 Body PDF Icon Dataset Icon
BABEL 2021 Body PDF Icon Dataset Icon
KIT-ML 2016 Body PDF Icon Dataset Icon
Audio-Conditioned Motion Datasets MOSA 2024 Body PDF Icon Dataset Icon
AIOZ-GDANCE 2023 Body PDF Icon Dataset Icon
BEAT 2022 Body + Face PDF Icon Dataset Icon
PhantomDance 2022 Body PDF Icon Dataset Icon
AIST++ 2021 Body PDF Icon Dataset Icon
Trinity 2020 Body + Face PDF Icon Dataset Icon
AniDance 2018 Body PDF Icon Dataset Icon
Speech-Conditioned Motion Datasets CHDTF 2025 Face PDF Icon Dataset Icon
Hallo3 2024 Body PDF Icon Dataset Icon
MultiTalk 2024 Body PDF Icon Dataset Icon
ZEGGS 2023 Body + Face PDF Icon Dataset Icon
BEAT 2022 Body + Face PDF Icon Dataset Icon
CelebV-HQ 2022 Face PDF Icon Dataset Icon
VFHQ 2022 Face PDF Icon Dataset Icon
ViCo 2022 Face PDF Icon Dataset Icon
TalkHead-1KH 2021 Face PDF Icon Dataset Icon
MEAD 2020 Face PDF Icon Dataset Icon
PATS 2020 Body + Face Paper 1 Paper 2 Dataset 1 Dataset 2
Trinity 2020 Body + Face PDF Icon Dataset Icon
Speech2Gesture 2019 Body + Face Paper Link Dataset Icon
CelebV 2018 Face Paper Link Dataset Link
VoxCeleb2 2018 Face Paper Link Dataset Link
VoxCeleb1 2017 Face Paper Link Dataset Link
LRS 2017 Face Paper Link Dataset Link
MV-LRS 2017 Face Paper Link Dataset Icon
LRW 2016 Face Paper Link Dataset Link
Scene-Conditioned Motion Datasets Habitat 2023 Body Paper 1 Paper 2 Paper 3 Dataset 1 Dataset 2 Dataset 3
Circle 2023 Body Paper Link Dataset Link
HUMANISE 2022 Body Paper Link Dataset Link
COUCH 2022 Body Paper Link Dataset Link
SAMP 2021 Body Paper Link Dataset Link
GTA-IM 2020 Body + Face Paper Link Dataset Link
PROX 2019 Body Paper Link Dataset Link
JTA 2018 Body Paper Link Dataset Link
PiGraph 2016 Body Paper Link Dataset Link
Action-Conditioned Motion Datasets Motion-X 2023 Body PDF Icon Dataset Icon
BABEL 2021 Body PDF Icon Dataset Icon
EMOGAIT 2021 Body Paper Link Dataset Icon
HuMMan 2021 Body Paper Link Dataset Link
HumanAct12 2020 Body Paper Link Dataset Link
NTU-RGB+D 2016 Body Paper 1 Paper 2 Dataset 1 Dataset 2
Penn Action 2013 Body PDF Icon Dataset Icon
UCF101 2012 Body Paper Link Dataset Link
General Motion Capture Datasets BioCV 2024 Body Paper Link Dataset Link
Motion-X 2023 Body PDF Icon Dataset Icon
HuMMan 2021 Body Paper Link Dataset Link
MoVi 2021 Body Paper Link Dataset Link
AMASS 2019 Body Paper Link Dataset Link
Human3.6M 2014 Body Paper Link Dataset Link
Interaction Datasets NoXi-J 2024 Body Paper Link Dataset Link
InterHuman 2024 Body Paper Link Dataset Link
Audio2Photoreal 2024 Body + Face Paper Link Dataset Link
RealTalk 2023 Face Paper Link Dataset Link
L2L 2023 Face Paper Link Dataset Link
GRAB 2020 Body Paper Link Dataset Link
NoXi 2017 Body Paper Link Dataset Link


Generative Techniques

Roadmap of Motion Generation Techniques.


Facial Animation Methods
Model Approach Year Paper GitHub
Diffusion AV-Flow 2025 Paper Link GitHub Logo
AniPortrait 2024 Paper Link GitHub Logo
DAWN 2024 Paper Link GitHub Logo
EDTalk 2024 Paper Link GitHub Logo
EMO: Emote Portrait Alive 2024 PDF Icon GitHub Logo
MEMO 2024 Paper Link GitHub Logo
Real3D-Portrait 2024 Paper Link GitHub Logo
EAT-Face 2024 Paper Link GitHub Logo
GAN ToonifyGB 2025 Paper Link GitHub Logo
G3FA 2024 Paper Link GitHub Logo
Style2Talker 2024 Paper Link GitHub Logo
VideoReTalking 2022 Paper Link GitHub Logo
EAMM 2022 Paper Link GitHub Logo
Talking Face Generation 2019 Paper Link GitHub Logo
Neural Network and VAE-based EmoHuman 2025 Paper Link GitHub Logo
CustomListener 2024 Paper Link GitHub Logo
Talk3D 2024 Paper Link GitHub Logo
FlowVQTalker 2024 Paper Link GitHub Logo
FreeAvatar 2024 Paper Link GitHub Logo
Can Language Models Learn to Listen? 2023 Paper Link GitHub Logo
MODA 2023 Paper Link GitHub Logo
SadTalker 2023 Paper Link GitHub Logo
VividTalk 2023 Paper Link GitHub Logo
Learning2Listen 2022 Paper Link GitHub Logo
RLHG 2022 Paper Link
ELP 2023 Paper Link
Trans-VAE 2023 Paper Link GitHub Logo
EVP 2021 Paper Link GitHub Logo


Body Animation Methods
Model Approach Year Paper GitHub
Diffusion Goal-Driven Motion Synthesis 2025 Paper Link
Light-T2M 2025 Paper Link GitHub Logo
AMUSE 2025 Paper Link GitHub Logo
SkeletonDiffusion 2025 Paper Link GitHub Logo
UniMuMo 2025 Paper Link GitHub Logo
EMDM 2024 Paper Link GitHub Logo
FlowMDM 2024 PDF Icon GitHub Logo
MotionDiffuse 2024 PDF Icon GitHub Logo
MoFusion 2023 PDF Icon GitHub Logo
PhysDiff 2023 PDF Icon
GestureDiffuCLIP 2023 PDF Icon
GAN Conditional GAN for Enhancing Diffusion Models 2025 PDF Icon
MoDI 2023 PDF Icon GitHub Logo
BelFusion 2023 PDF Icon GitHub Logo
ActFormer 2023 PDF Icon GitHub Logo
BiHMP-GAN 2019 PDF Icon
Neural Network and VAE-based Audio2Moves 2025 PDF Icon
MotionGPT 2024 PDF Icon GitHub Logo
MoMask 2024 PDF Icon GitHub Logo
SATO 2024 PDF Icon GitHub Logo
M3GPT 2024 PDF Icon GitHub Logo
PhysMoP 2024 PDF Icon GitHub Logo
Fg-T2M 2023 PDF Icon
T2M-GPT 2023 PDF Icon GitHub Logo
MotionBERT 2023 PDF Icon GitHub Logo
MotionClip 2022 PDF Icon GitHub Logo
PoseGPT 2022 PDF Icon GitHub Logo
PoseScript 2022 PDF Icon GitHub Logo


Paper

L. R. Sookha, N. Pakhale, M. Ganaie, A. Dhall
A Survey of Body and Face Motion Animation: Datasets, Metrics and Generative Techniques

[Bibtex]


Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.