Generating LoRA weights per input instead of training one adapter
- Source
- x.com
- Date
One static model does not fit all😭 We just dropped our latest work: Functional Neural Memory. Instead of static models, we generate custom "parameters" for every single input. ✅Prompt your model anytime ✅Instant personalization ✅Better instruction following ✅Flexible & dynamic memory (w/o memory bank✌️) (🧵1/6)
🧠Our method is inspired by two key observations in model optimization: 1️⃣Shared LoRA/SFT is forced into compromise when optimizing heterogeneous objectives. 2️⃣Single LoRA targeting specific domain generalizes poorly. (🧵2/6)
We propose HY-WU: a scalable framework for on-the-fly conditional generation of low-rank (LoRA) updates. HY-WU synthesizes instance-conditioned adapter weights from hybrid image–instruction representations and injects them into a frozen backbone during the forward pass, producing instance-specific operators without test-time optimization. (🧵3/6)
🎨Showcases of HY-WU's personalization ability, now you can try our cute dolls and cool clothes. (🧵4/6)
Context
Tencent's HY-WU explores adapting an image-editing model separately for each request. It reads the image together with the editing instruction, generates a compact set of weight changes called a adapter, and applies those changes while the main model processes the request. The main model's underlying weights stay fixed, according to the project description.
This gives each input its own adjustment without running a new procedure for that request. The project calls this approach functional neural memory: a lightweight generator produces a conditioned adapter for each request. The described application is text-guided image editing, where the desired change depends on both the picture and the user's instruction.
- LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
articleExplorationBench: Feedback and Self-Designed Probes Drive Agent Discovery
postHy Image3.5 preview is live. 🚀 Professional-grade image generation, +30% win…
postTencent Hunyuan's EvolveScaler trips up frontier models on evolving facts
postTencent Hunyuan open-sources AuK, a unified speech generation and editing model
Checking sign-in…
Loading comments…