- Category
- AI Tools
- Rank
- No. 1020Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- bytedance
- GitHub
- 674 stars
- Date
About
High-fidelity face-swapping for video using a diffusion transformer, bridging the image-to-video gap from ByteDance research.
What it does
DreamID-V replaces the face in an existing video using a cropped reference portrait. It derives face masks and pose guidance from the source footage, encodes those inputs with the video model, then writes the generated frames as an MP4.
Why it's ranked here
This is credible research code with several practical inference routes, not a finished application. It supports automatic pose preprocessing, lower-step inference, CPU offloading, and distributed execution. Those capabilities make it worth testing, while the manual model setup and script-level interface limit broader adoption.
What's good
The repository covers more than the core model. It can generate pose and face-mask guidance from input footage, offers MediaPipe and DWPose variants, supports two sampling solvers, preserves reproducibility through seeds, and scales across GPUs using model sharding and sequence parallelism.
Tradeoffs
Setup requires separate DreamID-V, Wan, and pose-estimation checkpoints placed correctly. The pipeline targets CUDA, and distributed execution uses NCCL. There is no documented service API or graphical interface in this repository. Although a prompt option is exposed, the generation scripts replace it with a fixed placeholder.
How to use it well
It suits researchers and experienced ML engineers running controlled, consent-based video experiments on GPU infrastructure. Start with a cropped square face image, use the DWPose route for stronger pose extraction, and lower sampling steps for simple scenes. It does not cover deployment, moderation, consent management, or production API serving.
Technical notes+
requirements.txt installs PyTorch 2.4 or newer, Diffusers 0.35.2, Transformers 4.49.0, xFuser 0.4.4, MediaPipe 0.10.5, ONNX Runtime, OpenCV, and NumPy below 2. generate_dreamidv.py builds pose and mask videos with MediaPipe, while generate_dreamidv_dwpose.py and generate_dreamidv_faster.py call process_dwpose from pose/extract.py. dreamidv_wan/wan_swapface.py exposes DreamIDV, loads T5, VAE, and DiT checkpoints, targets cuda:{device_id}, and accepts video, mask, portrait, and pose inputs. The launchers support FSDP plus xDiT Ulysses and ring parallelism, with NCCL for distributed execution. Source headers state Apache License 2.0.
Observed
- License
- Apache License 2.0 stated in Python source headers
- Primary language
- Python
- Installation
- Dependencies installed from requirements.txt with pip; model checkpoints downloaded separately
- Interface
- Command-line inference scripts and an importable Python pipeline class
- Acceleration
- CUDA execution with optional CPU model offloading
- Distributed support
- Multi-GPU inference through NCCL, FSDP, and xDiT sequence parallelism
- Output
- Generated video is written as MP4
Read from README.md, requirements.txt, generate.py, generate_dreamidv.py, generate_dreamidv_dwpose.py, generate_dreamidv_faster.py, assets/index.js, pose/extract.py, assets/bootstrap.min.js, dreamidv_wan/__init__.py, express_adaption/utils.py, dreamidv_wan/wan_swapface.py.
What it can do
Swap faces in video content
Source video and target face image → Video with swapped faces
Generate high-fidelity facial transformations
Original face and replacement face data → Realistic face-swapped content
Bridge image-to-video conversion for faces
Static face image → Video-compatible face representation
Apply diffusion transformer processing to faces
Face image or video frames → Enhanced face data for swapping
Process video frames for face replacement
Video file with faces to be replaced → Frame-by-frame face-swapped video
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.