Vibeleaderboard
← All Intel
Intel / post

StepFun opens Step 3.5 Flash base weights, training code and SFT data

Source
x.com
Date
StepFun@StepFun_ai
Thread · 4 parts

"can we get the base model?" sure. here's two. "can we get the code?" sure. here's SteptronOSS. "what about the SFT data?" coming soon. maximum sincerity, minimum barriers. - Step 3.5 Flash Base — pretrained foundation - Step 3.5 Flash Base-Midtrain — code, agents & long-context - SteptronOSS — open-sourced, ready for your custom workflows - SFT Data — coming soon for reference not just the final checkpoint — a customizable pipeline. 🤗 https://t.co/pVaFiJ87UT 🤗 https://t.co/YDOZUcqgXh 💻 https://t.co/G42LCHmQMr

now on @ModelScope2022 too 👇 Base: https://t.co/w8w0vwVbpt Midtrain: https://t.co/Avf21PDf6H

the last missing piece. our SFT training dataset is now available at https://t.co/S7M3KCswxH https://t.co/3FZbwpAeSj

our SFT dataset is now on @ModelScope2022 too! https://t.co/F6nZVIKScu https://t.co/BrrVexFrPr

Context

StepFun released the full pipeline behind Step 3.5 Flash rather than only the finished chat model: a pretrained base checkpoint, a second midtrain checkpoint built for code, agents and long- work, and SteptronOSS, the open-sourced training code. A base checkpoint is the model before it has been tuned to follow instructions or chat, so releasing it lets outside teams run their own post-training rather than start from, and be limited by, the choices StepFun made when tuning the final chat model.

In follow-up posts over the following weeks, StepFun also published the supervised (SFT) dataset used to tune the model, first on Hugging Face and later mirrored to ModelScope. Releasing base weights, training code and SFT data together is a broader disclosure than most releases, which typically stop at the final checkpoint; this is StepFun's own description of what it released, without independent confirmation of dataset completeness.

Terms in this piece · Glossary
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
More from StepFun
Recommended reads
Comments

Checking sign-in…

Loading comments…