Fireworks Forge: faster agent inference, cheaper background runs, Nexus routing
- Source
- fireworks.ai
- Author
- Fireworks
- Date
Fireworks used its Forge event to announce a set of inference and training updates. The headline number comes from a new ultra-fast tier, offered in preview at about 1,500 tokens per second, where Fireworks says a 30-turn agent task fell from one minute to eight seconds. Speed matters more for agents than for chat because each turn waits on the previous one, so per-token latency compounds across a long run. The same announcement covers the other end of the spectrum. A background mode charges half price for work that does not need an immediate answer, and prompt caching now lasts multiple days instead of expiring quickly. Fireworks also announced Nexus, which routes between models inside coding apps, and serverless LoRA serving for fine-tuned adapters. Taken together, a team can now place fast interactive agents, overnight batch jobs and fine-tuned variants on separate price and latency points from one provider. The figures are Fireworks' own, and the fast tier is still a preview.
Context
Fireworks' Forge launches price agent inference by how urgently the work is needed (announcement). Fireworks, which serves open-weight models for companies, starts from the observation that an agent often makes many model calls to finish one task, so each call's delay adds to the wait and running more agents at once adds cost. Its answer is two runtimes. Ultra-fast, in public preview on NVIDIA Blackwell GPUs, reaches up to 1,500 tokens per second, as demonstrated on DeepSeek V4.1 Flash; in a benchmarked 30-turn agent task, Fireworks says completion time fell from 1 minute to 8 seconds. Background Mode runs eligible work that can wait, such as generating data for evaluations or overnight research, at half price.
Other pieces of the launch address the same cost and capacity pressures. Multi-day lowers the cost of repeated context, and higher serverless rate limits and Elastic Burst, both generally available, let more agents run at once. Fireworks' own model Ember-1 used about 40% fewer tokens than Kimi K3 at comparable quality in Fireworks' evaluations; Vercel's listing when Ember-1 reached its AI Gateway describes it as a built on Kimi K3.
The rest of the launch is about companies running models they shape and control. Nexus brings open models into the Codex app, Claude Desktop and OpenCode alongside people's existing AI subscriptions, with Pro plans from $8 a month and Teams plans from $10 per seat a month. No Eval, in private preview, turns production traces into task-specific evaluations. Specialized Model Inference, in public preview, serves trained adapters (small sets of added weights that customize a base model), starting with GLM 5.3 Flash, at per-token prices with no charge while idle. Fireworks Sovereign, in private preview, runs Fireworks' inference engine in a customer's own cloud or data center, and Fireworks says it already manages more than 25,000 GPUs in customer environments.
All of the performance figures are Fireworks' own, and several of the new products are previews rather than generally available services.
- Fireworks Ultra-fast, running on NVIDIA Blackwell GPUs and now in public preview, reaches up to 1,500 tokens per second on DeepSeek V4.1 Flash; Fireworks says a benchmarked 30-turn agent task fell from 1 minute to 8 seconds.
- Background Mode runs eligible work that can wait, such as eval data generation or overnight research, at half price, and multi-day prompt caching lowers the cost of repeated context.
- Fireworks says its Ember-1 model used about 40% fewer tokens than Kimi K3 at comparable quality in its own evaluations.
- Nexus Desktop brings open and closed into the Codex app, Claude Desktop and OpenCode with centralized spending controls; Pro plans start at $8 per month and Teams at $10 per seat per month.
- Specialized Model Inference serves LoRA adapters serverlessly with per-token pricing and no charge while idle, starting with GLM 5.3 Flash, while No Eval (private preview) turns production traces into task-specific evals.
- prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
- reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
- LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
- model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
Checking sign-in…
Loading comments…