Intel / article

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

Source: together.ai
Author: Together AI
Date: Published Jun 2, 2026

Context

MiniMax M3 selects relevant blocks of for , but serving that design efficiently still requires careful data movement. Together AI reorganized one computation so a stored context block can serve several queries after being loaded, reducing repeated transfers. It also adapted existing attention kernels to the model's selected blocks. The engineering account explains these changes.

Across its combined serving optimizations, Together reports 81–125% higher throughput at different concurrency levels on -style traffic. That measures the serving stack's improvement, not a change in model accuracy. At the time of the report, Together was serving M3 for MiniMax, while public and a direct developer endpoint were still forthcoming.

Terms in this piece · Glossary

context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.

Key quotes

“The most novel architectural change in M3 is MiniMax Sparse Attention (MSA), which is designed to address the attention-computation bottleneck seen in MiniMax M2.7. Its block-sparse attention mechanism caps the maximum number of tokens each query can attend to, reducing the cost of long-context processing and making much longer context windows practical.”
Together AI

“The nature of the block sparse attention allows multiple queries to attend to the same key-value blocks. Thus, if we iterate each query to calculate attention with key-value blocks, we are duplicating the KV movement from HBM to SRAM on GPU.”
Together AI

“This design unblocked us to use the existing attention kernels that support GQA without having to rewrite a new one that supports sparse attention from scratch. Because the selected blocks for each query is limited, the kernels to find mapping between blocks to pages are with very low overhead. This design gives us 5% improvement on the decode throughput.”
Together AI

“Image and video inputs need a fair amount of CPU work before they are useful to a vision encoder: downloading, decoding, frame sampling, resizing, and converting into patch tensors. Doing this inside the inference engine ties up resources that should be spent on generation.”
Together AI

“Since receiving the MiniMax M3 weights and model architecture, we strived to improve the inference performance. We have reached 81% - 125% increase on various concurrency levels on common agentic shape traffic.”
Together AI

Read the source www.together.ai