Today we release LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: bidirectional encoders that stay fast at long context, even on CPU.
> LFM2.5-Encoder-230M: about 3.7x faster than ModernBERT-base on CPU at 8,192 tokens. Under 30s per forward pass, versus over a minute and a half.
> LFM2.5-Encoder-350M: 4th of 14 models on GLUE, SuperGLUE, and multilingual classification, behind only three larger models, one of them nearly 10x its size.
🧵
Two sizes in the same family:
> LFM2.5-Encoder-350M: the higher-accuracy pick.
> LFM2.5-Encoder-230M: smaller and faster, for tighter hardware or higher throughput.
Both are pre-trained with a masked-language objective, so you fine-tune them for classification, token-level tasks, or retrieval.
(2/6)
Built on the LFM2 hybrid backbone, adapted from causal decoders into bidirectional encoders: bidirectional attention, non-causal short convolutions, and a 30% mask rate. The backbone's cost grows slowly as inputs get longer.
> Context: 8,192 tokens, about 13 to 15 pages in one forward pass.
> Runs on CPU with no GPU, where the speed edge is largest.
> On GPU, the same pattern holds at the longest inputs.
(3/6)
See it in action. We built demos that run entirely on CPU:
> Prompt routing: https://t.co/Y7CfhQrSTy
> Policy linting: https://t.co/DPu9iV8mbh
> Spell checker: https://t.co/92NrnWDsEo
Bonus: we even run the encoder as a masked-diffusion chatbot. https://t.co/4EIW6sYZiF
(4/6)
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
Classification, routing and retrieval that run on CPU get an encoder that stays fast at 8,192 tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →, roughly 3.7x quicker than ModernBERT-base at that length, ready to fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → for token-level and retrieval tasks.
Read the full thread on X ↗
