Fernando Fernandes Neto researches efficient adaptation of language models. He co-authored Spectrum, which selects model layers for training, and work on transplanting tokenizers between models. Arcee AI also credits him on its foundation-model releases.[2][3][4]
Start with Spectrum implementation →
Contributions
Select the layers worth updating
Spectrum scores modules by signal-to-noise ratio, trains a selected fraction and freezes the rest. Fernandes Neto co-authored the paper. Its results concern the evaluated tasks.[1]
Turn a scan into a training configuration
The Spectrum implementation scans a checkpoint, ranks modules and writes a configuration listing the parameters to unfreeze. That connects an analysis of existing weights to an actual fine-tuning run. The repository supplies integration with Axolotl and permits reuse of previously saved scans.[2]
Move between different model vocabularies
A separate paper by Fernandes Neto and Charles Goddard develops training-free tokenizer transplantation. Orthogonal Matching Pursuit finds a sparse mapping between representations so a model can use another tokenizer’s vocabulary without a full retraining cycle. The method addresses incompatible token vocabularies, a concrete obstacle when adapting or combining models.[3]
Tools, writing & talks
Tools
Start here
Spectrum implementation
github.comA scanner that produces ranked modules and a configuration for selective training.
Inspect a saved scan and the generated unfreezing configuration before running a new scan. Python, PyTorch and model-training experience help. Large checkpoints need substantial memory; begin with a model that fits your available hardware.
Research collaborator[2]
Writing, research & talks
- Tokenizer transplantationarxiv.org
The researchers’ explanation of transferring a tokenizer through a sparse representation mapping.
Co-author[3]