Use Mergekit To Extract Lora Adapters From Any Fine Tuned Model

Arcee AI's MergeKit can package the changes made by fine-tuning as a compact LoRA adapter, according to an April 2024 Arcee post. It compares a fine-tuned model with its base model, then approximates the differences in their parameter values using a smaller low-rank representation. Arcee says that, with an adequate rank setting, fine-tunes shrink from gigabytes to megabytes, which makes them easier to share and merge. Extraction announcement.

The approximation matters: a smaller adapter need not preserve every capability of the original fine-tune. Arcee's experiments on one Mistral-based model explore different adapter sizes, while its proposed library of adapters would let developers select and combine task-specific changes. Dynamic combinations are presented as a possible use of the tool. Mechanism and proposed applications.