Echomimic v2

<h1 align='center'>EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation</h1> <div align='center'> <a href='https://github.com/mengrang' target='_blank'>Rang Meng</a>¹&emsp; <a href='https://github.com/' target='_blank'>Xingyu Zhang</a>&emsp; <a href='https://lymhust.github.io/' target='_blank'>Yuming Li</a>²&emsp; <a href='https://openreview.net/profile?id=~Chenguang_Ma3' target='_blank'>Chenguang Ma</a>² </div> <div align='center'> Terminal Technology Department, Alipay, Ant Group. </div> <p align='center'> ¹Core Contributor&emsp; ²Corresponding Authors </p> <div align='center'> <a href='https://antgroup.github.io/ai/echomimic_v2/'><img src='https://img.shields.io/badge/Project-Page-blue'></a> <a href='https://huggingface.co/BadToBest/EchoMimicV2'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Model-yellow'></a> <!--<a href='https://antgroup.github.io/ai/echomimic_v2/'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Demo-yellow'></a>--> <a href='https://modelscope.cn/models/BadToBest/EchoMimicV2'><img src='https://img.shields.io/badge/ModelScope-Model-purple'></a> <!--<a href='https://antgroup.github.io/ai/echomimic_v2/'><img src='https://img.shields.io/badge/ModelScope-Demo-purple'></a>--> <a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a> <a href='https://openaccess.thecvf.com/content/CVPR2025/papers/Meng_EchoMimicV2_Towards_Striking_Simplified_and_Semi-Body_Human_Animation_CVPR_2025_paper.pdf'><img src='https://img.shields.io/badge/Paper-CVPR2025-blue'></a> <a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a> </div> <div align='center'> <a href='https://github.com/antgroup/echomimic_v2/discussions/53'><img src='https://img.shields.io/badge/English-Common Problems-orange'></a> <a href='https://github.com/antgroup/echomimic_v2/discussions/40'><img src='https://img.shields.io/badge/中文版-常见问题汇总-orange'></a> </div>

🚀 EchoMimic Series

• EchoMimicV1: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning. GitHub
• EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. GitHub
• EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animation. GitHub

📣 Updates

• [2025.08.09] 🔥🔥 We update the EchoMimicV3 and release the code.
• [2025.02.27] 🔥 EchoMimicV2 is accepted by CVPR 2025.
• [2025.01.16] 🔥 Please check out the discussions to learn how to start EchoMimicV2.
• [2025.01.16] 🚀🔥 GradioUI for Accelerated EchoMimicV2 is now available.
• [2025.01.03] 🚀🔥 One Minute is All You Need to Generate Video. Accelerated EchoMimicV2 are released. The inference speed can be improved by 9x (from ~7mins/120frames to ~50s/120frames on A100 GPU).
• [2024.12.16] 🔥 RefImg-Pose Alignment Demo is now available, which involves aligning reference image, extracting pose from driving video, and generating video.
• [2024.11.27] 🔥 Installation tutorial is now available. Thanks AiMotionStudio for the contribution.
• [2024.11.22] 🔥 GradioUI is now available. Thanks @gluttony-10 for the contribution.
• [2024.11.22] 🔥 ComfyUI is now available. Thanks @smthemex for the contribution.
• [2024.11.21] 🔥 We release the EMTD dataset list and processing scripts.
• [2024.11.21] 🔥 We release our EchoMimicV2 codes and models.
• [2024.11.15] 🔥 Our paper is in public on arxiv.

🌅 Gallery

Introduction

<table class="center"> <tr> <td width=50% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/f544dfc0-7d1a-4c2c-83c0-608f28ffda25" muted="false"></video> </td> <td width=50% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/7f626b65-725c-4158-a96b-062539874c63" muted="false"></video> </td> </tr> </table>

English Driven Audio

<table class="center"> <tr> <td width=100% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/3d5ac52c-62e4-41bc-8b27-96f005bbd781" muted="false"></video> </td> </tr> </table> <table class="center"> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/e8dd6919-665e-4343-931f-54c93dc49a7d" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/2a377391-a0d3-4a9d-8dde-cc59006e7e5b" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/462bf3bb-0af2-43e2-a2dc-559e79953f3c" muted="false"></video> </td> </tr> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/0e988e7f-6346-4b54-9061-9cfc7a80e9c8" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/56f739bd-afbf-4ed3-ab15-73a811c1bc46" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/1b2f7827-111d-4fc0-a773-e1731bba285d" muted="false"></video> </td> </tr> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/a76b6cc8-89b9-4f7e-b1ce-c85a657b6dc7" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/bf03b407-5033-4a30-aa59-b8680a515181" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/f98b3985-572c-499f-ae1a-1b9befe3086f" muted="false"></video> </td> </tr> </table>

Chinese Driven Audio

<table class="center"> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/a940a332-2fd1-48e7-b3c4-f88f63fd1c9d" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/8f185829-c67f-45f4-846c-fcbe012c3acf" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/a49ab9be-f17b-41c5-96dd-20dc8d759b45" muted="false"></video> </td> </tr> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/1136ec68-a13c-4ee7-ab31-5621530bf9df" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/fc16d512-8806-4662-ae07-8fcf45c75a83" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/f8559cd1-f555-4781-9251-dfcef10b5b01" muted="false"></video> </td> </tr> <tr> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/c7473e3a-ab51-4ad5-be96-6c4691fc0c6e" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/ca69eac0-5126-41ee-8cac-c9722004d771" muted="false"></video> </td> <td width=30% style="border: none"> <video controls loop src="https://github.com/user-attachments/assets/e66f1712-b66d-46b5-8bbd-811fbcfea4fd" muted="false"></video> </td> </tr> </table>

⚒️ Automatic Installation

Download the Codes

bashCopy
  git clone https://github.com/antgroup/echomimic_v2
  cd echomimic_v2

Automatic Setup

• CUDA >= 11.7, Python == 3.10

bashCopy
   sh linux_setup.sh

⚒️ Manual Installation

Download the Codes

bashCopy
  git clone https://github.com/antgroup/echomimic_v2
  cd echomimic_v2

Python Environment Setup

• Tested System Environment: Centos 7.2/Ubuntu 22.04, Cuda >= 11.7
• Tested GPUs: A100(80G) / RTX4090D (24G) / V100(16G)
• Tested Python Version: 3.8 / 3.10 / 3.11

Create conda environment (Recommended):

bashCopy
  conda create -n echomimic python=3.10
  conda activate echomimic

Install packages with pip

bashCopy
  pip install pip -U
  pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 xformers==0.0.28.post3 --index-url https://download.pytorch.org/whl/cu124
  pip install torchao --index-url https://download.pytorch.org/whl/nightly/cu124
  pip install -r requirements.txt
  pip install --no-deps facenet_pytorch==2.6.0

Download ffmpeg-static

Download and decompress ffmpeg-static, then

export FFMPEG_PATH=/path/to/ffmpeg-4.4-amd64-static

Download pretrained weights

shellCopy
git lfs install
git clone https://huggingface.co/BadToBest/EchoMimicV2 pretrained_weights

The pretrained_weights is organized as follows.

./pretrained_weights/
├── denoising_unet.pth
├── reference_unet.pth
├── motion_module.pth
├── pose_encoder.pth
├── sd-vae-ft-mse
│   └── ...
└── audio_processor
    └── tiny.pt

In which denoising_unet.pth / reference_unet.pth / motion_module.pth / pose_encoder.pth are the main checkpoints of EchoMimic. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:

• sd-vae-ft-mse
• audio_processor(whisper)

Inference on Demo

Run the gradio:

bashCopy
python app.py

Run the python inference script:

bashCopy
python infer.py --config='./configs/prompts/infer.yaml'

Run the python inference script for accelerated version. Make sure to check out the configuration for accelerated inference:

bashCopy
python infer_acc.py --config='./configs/prompts/infer_acc.yaml'

EMTD Dataset

Download dataset:

bashCopy
python ./EMTD_dataset/download.py

Slice dataset:

bashCopy
bash ./EMTD_dataset/slice.sh

Process dataset:

bashCopy
python ./EMTD_dataset/preprocess.py

Make sure to check out the discussions to learn how to start the inference.

📝 Release Plans

Status	Milestone	ETA
✅	The inference source code of EchoMimicV2 meet everyone on GitHub	21st Nov, 2024
✅	Pretrained models trained on English and Mandarin Chinese on HuggingFace	21st Nov, 2024
✅	Pretrained models trained on English and Mandarin Chinese on ModelScope	21st Nov, 2024
✅	EMTD dataset list and processing scripts	21st Nov, 2024
✅	Jupyter demo with pose and reference image alignmnet	16st Dec, 2024
✅	Accelerated models	3st Jan, 2025
🚀	Online Demo on ModelScope to be released	TBD
🚀	Online Demo on HuggingFace to be released	TBD

⚖️ Disclaimer

This project is intended for academic research, and we explicitly disclaim any responsibility for user-generated content. Users are solely liable for their actions while using the generative model. The project contributors have no legal affiliation with, nor accountability for, users' behaviors. It is imperative to use the generative model responsibly, adhering to both ethical and legal standards.

🙏🏻 Acknowledgements

We would like to thank the contributors to the MimicMotion and Moore-AnimateAnyone repositories, for their open research and exploration.

We are also grateful to CyberHost and Vlogger for their outstanding work in the area of audio-driven human animation.

If we missed any open-source projects or related articles, we would like to complement the acknowledgement of this specific work immediately.

📒 Citation

If you find our work useful for your research, please consider citing the paper :

@article{meng2024echomimicv2,
  title={EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation},
  author={Meng, Rang and Zhang, Xingyu and Li, Yuming and Ma, Chenguang},
  journal={arXiv preprint arXiv:2411.10061},
  year={2024}
}
@article{meng2025echomimicv3,
  title={Echomimicv3: 1.3 b parameters are all you need for unified multi-modal and multi-task human animation},
  author={Meng, Rang and Wang, Yan and Wu, Weipeng and Zheng, Ruobing and Li, Yuming and Ma, Chenguang},
  journal={arXiv preprint arXiv:2507.03905},
  year={2025}
}
@article{meng2026echotorrent,
  title={EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation},
  author={Meng, Rang and Wu, Weipeng and Yin, Yingjie and Li, Yuming and Ma, Chenguang},
  journal={arXiv preprint arXiv:2602.13669},
  year={2026}
}

🌟 Star History