ComfyUI HyperLoRA
https://github.com/bytedance/comfyui-hyperlora- Category
- AI Tools
- Rank
- No. 1487Tools index
Previous survey · No. 1494 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- bytedance
- GitHub
- 498 stars
- Date
About
ComfyUI node implementing HyperLoRA: parameter-efficient adaptive generation for portrait synthesis (CVPR 2025).
What it does
It turns one or more reference portraits into identity-focused LoRA weights for SDXL. An adaptive network reads CLIP and ArcFace features, then mixes learned bases into weights that guide personalized image generation without training separately for every person.
Why it's ranked here
A strong choice for portrait workflows that need fast personalization and editable results. It offers dedicated fidelity and editability models, reproducible example workflows, and standalone LoRA generation. Its value narrows sharply when model compatibility, commercial use, or simple installation matters most.
What's good
The identity and base components separate facial identity from clothing and background information. Users can choose fidelity or editability, combine the result with ControlNet or InstantID, and start further LoRA training from the generated weights. Fixed seeds and included inputs make the supplied workflows reproducible.
Tradeoffs
Setup requires separately downloading CLIP, InsightFace, and HyperLoRA assets into an exact folder structure. The example workflows also require two ComfyUI plugins. Compatibility varies across SDXL base models, small faces may need repair, and the supplied model weights allow only non-commercial sharing and adaptation.
How to use it well
Use it for research or non-commercial SDXL portrait pipelines where reference images must become reusable identity LoRAs quickly. Start from the fidelity or editability workflow, then add face repair or ControlNet when needed. It does not replace broad SDXL compatibility testing, commercial model licensing, or final small-face cleanup.
Technical notes+
__init__.py exports ComfyUI node mappings from hyperlora/nodes.py. That module loads CLIP, InsightFace, safetensors checkpoints, resamplers, projectors, and HyperLoRA modules, then uses ComfyUI model patching to apply generated weights. hyperlora/modules.py implements Perceiver-style attention resampling and basis mixing for query, key, value, and output projections. hyperlora/configs.py defines configurable encoder, resampler, identity-token, rank, and optional base-LoRA settings. standalone/generate_loras.py provides an argparse CLI that processes JPEG or PNG inputs and saves one safetensors LoRA per image. Dependencies are declared in both pyproject.toml and requirements.txt.
Observed
- Code license
- GNU General Public License version 3
- Model license
- CC BY-NC 4.0
- Primary language
- Python
- Packaging
- Python project metadata in pyproject.toml, with dependencies also listed in requirements.txt
- Interfaces
- ComfyUI custom nodes and a standalone command-line LoRA generator
- Core dependencies
- diffusers, insightface, onnxruntime, opencv-python, and transformers
Read from README.md, pyproject.toml, requirements.txt, __init__.py, hyperlora/nodes.py, hyperlora/common.py, hyperlora/configs.py, hyperlora/modules.py, hyperlora/node_fields.py, standalone/generate_loras.py, standalone/mock/folder_paths.py, standalone/mock/comfy/sd.py, standalone/mock/comfy/model_patcher.py, standalone/mock/comfy/model_management.py, LICENSE.
What it can do
Generate portrait images using parameter-efficient adaptation
Base model and adaptation parameters → Synthesized portrait images
Adapt pre-trained models for specific portrait styles
Pre-trained model and target style parameters → Adapted model weights
Process portrait synthesis requests through ComfyUI workflow
ComfyUI node configuration and parameters → Generated portrait outputs
Apply HyperLoRA parameter-efficient fine-tuning
Base neural network and training data → Fine-tuned model with minimal parameter overhead
Control portrait generation with adaptive parameters
Generation prompts and control parameters → Customized portrait variations
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.