Intel / post

Serving Step 3.7 Flash on Modal with SGLang and eight H100s

Source: x.com
Date: Published Jun 3, 2026

StepFun@StepFun_ai

Deploy Step 3.7 Flash on @modal with SGLang 🚀 Modal is a serverless AI platform for deploying and scaling compute-intensive workloads without managing infrastructure. Their new guide shows how to serve our open-weight Step 3.7 Flash with SGLang on Modal, using 8×H100 GPUs, Modal Volumes, and an OpenAI-compatible chat completions endpoint. Excited to collaborate with Modal to make StepFun models more accessible to builders. https://t.co/E7gaNPq02H

Jun 3, 2026 View on X

Context

Per StepFun's post, a new guide shows how to self-host Step 3.7 Flash on Modal, a serverless compute platform that manages the underlying infrastructure, using the SGLang engine across eight Nvidia H100 GPUs and Modal's persistent storage volumes, with the resulting service exposed through an OpenAI-compatible chat completions endpoint.

That hardware detail matters because Step 3.7 Flash is a 198-billion-parameter sparse model with roughly 11 billion active parameters per (model specifications); the OpenAI-compatible endpoint means a team already calling an OpenAI-style API can point at a self-hosted deployment largely by changing the base URL, rather than rewriting client code.

Terms in this piece · Glossary

inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.

Read the source x.com