Inference

Step 1: Clone this repository

git clone https://github.com/Orange-3DV-Team/MoCha.git
cd MoCha

Step 2: Set up the environment

# 1. Create conda environment
conda create -n MoCha python==3.10

# 2. Activate the environment
conda activate MoCha

# 3. Install pip dependencies
pip install -r requirements.txt

Step 3: Download the pretrained checkpoints

1. Download the pre-trained Wan2.1 models from huggingface
2. Download the pre-trained MoCha checkpoint

Please download from huggingface and place it in ./checkpoints.

Step 4: Test the example videos

python inference_mocha.py

Test your own video

To start your own character replacement with MoCha, the following three inputs are required:

• Source Video: The original video with the character to be replaced.
• Designation Mask for the First Frame: A mask marking the source character to be replaced in the first frame of Source Video.
• Reference Images: Reference Images of the new character for replacement with clean background. We recommend uploading at least one high-quality, front-facing facial close-up.

https://github.com/user-attachments/assets/3ae70cc1-9bf2-47ca-b6c2-bf7c84f0054f

Then organize your test data following the structure of the ./data/test_data.csv.

• source_video: Path to Source Video.
• source_mask: Path to Designation Mask.
• reference_1: Path to first Reference Image.
• reference_2: Path to second Reference Image. This image needs to be a high-quality, front-facing facial close-up. (You can even zoom up your first reference image!) If you really cannot provide this reference image, leave it as None.

Finally, test your videos by:

python inference_mocha.py --data_path path/to/your/data.csv