transformers v5.24.0 — Release v5.24.0
- Source
- github.com
- Author
- Cyrilvallez
- Date

Context
The change in transformers v5.24.0 that most developers will notice is a rename of one of the library's most-used arguments (GitHub release notes). transformers is Hugging Face's Python library for loading and running open models. When a model generates text, it saves intermediate state from the tokens it has already processed so it does not recompute it for every new token, and that saved state is the cache. -based models called it past_key_values, while Mamba-style and linear-attention models called it cache_params, so hybrid models that mix both kinds of layer had to handle two names for one object. From this release it is called cache everywhere: in model calls, in generate, and in every model output.
Existing code keeps working for now. The old names are converted automatically with a one-time deprecation warning; model forward calls accept them until v5.40.0 and generate until v6.0.0, and outputs still answer to the old names. Passing an old name and cache together raises an error. For the rename, the people who need to act are authors of custom models defined outside the library, which should rename their arguments, and anyone with a custom Cache subclass that overrides update, which needs to accept an extra kwargs argument. Projects using the PEFT library also need peft 0.22.0 or later. The release flags two other breaking changes with no grace period: generative multimodal models now return shared output classes, so code that checks for a model-specific output class must switch to them, and the long-deprecated modeling_attn_mask_utils module is gone, so remote-code models that import from it must move to transformers.masking_utils.
The release also tightens one security default. Exported models saved as pickled .pt2 programs, a format that can run arbitrary code when loaded, now require trust_remote_code=True, so loading one from someone else becomes an explicit choice; ONNX, OpenVINO and ExecuTorch exports load without that flag. Alongside these changes the release adds support for new models, including Thinking Machines Lab's Inkling 2 Mini and Meta FAIR's Omnilingual ASR speech-recognition models, which cover more than 1,600 languages.
- Inkling 2 Mini cuts Inkling 2 Small's 48 decoder layers to 24 and compensates by running layers 6 to 17 three times, with a learned message-passing step that mixes the previous pass's residuals into each loop layer's attention and expert inputs.
- v5.24.0 renames past_key_values and cache_params to one cache argument everywhere, so hybrid models stop juggling two names. Old names still work with a warning until v5.40.0 for forward and v6.0.0 for generate; passing both raises a ValueError.
- The cache rename requires peft 0.22.0 or later. Custom Cache subclasses that override update should accept a new kwargs argument, because layers now call cache.update(key_states, value_states, layer_idx, kwargs=kwargs).
- Pickled .pt2 exported programs now need trust_remote_code=True because loading them can run arbitrary code, while exported ONNX, OpenVINO and ExecuTorch programs now load without that flag.
- NVIDIA's Dory, also added in this release, is a looped normalized-GPT model that reapplies a shared block of layers; its loop count can be changed at inference time on the same checkpoint to trade speed against quality.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
- Field — One named piece of a record, such as name or approved_on; a record is one entity's set of values, one per field.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
- Multimodality — The ability to understand or generate more than one kind of media.
Checking sign-in…
Loading comments…


