together ai
Together AI (@togethercompute) is a San Francisco-based AI infrastructure company, not an individual, founded in 2022 by Vipul Ved Prakash (CEO), Ce Zhang, Chris Ré, Tri Dao, and Percy Liang. It operates an "AI Native Cloud" platform for training, fine-tuning, and running generative AI models, and is known for research contributions and advocacy for open-source AI, including datasets like RedPajama. The company has raised over $1 billion in funding from investors including Nvidia and General Catalyst, reaching a multi-billion-dollar valuation, and counts enterprise customers such as Zoom and Salesforce.
Notable people
People map
Current peopleNotable public-facing people we cover. Grouped by public roles, not reporting lines; this is not a complete staff directory.
Leadership
- Percy Liang
Founder
Research
- Simran Arora
SVP of Research
- Tri Dao
Co-founder and Chief Scientist · Jul 2023 – Present
Other public roles
- Hassan El Mghari
Developer Experience lead
Writing, research & talks
We ran 452 DeepSWE rollouts on Kimi K3 and Claude Fable 5. Fable leads pass@1 by 1.4 points; Kimi K3 wins pass@4 and delivers 2.8x the solves per dollar.
- What does 99.9% uptime mean for inference?together.ai
Reliability numbers are easy to publish. We break down what 99%, 99.9%, and 99.99% uptime actually require, the failure domains each tier has to survive, and the questions to ask any inference provider before you commit.
ParallelKernelBench tests whether LLMs can write fast multi-GPU CUDA kernels across 87 real workloads. The best model solves under a third, but a few generated kernels beat any public implementation.
We generated 12 landing pages with Kimi K2.7 Code and Claude Fable 5. Kimi cost 94% less and scored within a few points on every page. Here's what actually moved the needle.
How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway.
Together AI built the fastest speech-to-text stack on Artificial Analysis by treating ASR as a full-path systems problem, not just a GPU inference problem.
Inference benchmarks framed around coding-agent traffic rather than synthetic prompts: 31% more tokens per second than TensorRT-LLM, about 2x better time-to-first-token at saturation, and 76% lower cost than Claude Opus 4.6. Vendor-run, so check the methodology before budgeting on it.
DeepSeek-V4 makes million-token context a serving-systems problem. Together AI explores the inference work behind V4 on NVIDIA HGX B200, including compressed KV layouts, prefix caching, kernel maturity, and endpoint profiles for long-context workloads.
Rollout is the silent bottleneck in RL post-training. DAS fixes it with adaptive speculative decoding — up to 50% faster, zero degradation in reward quality.
Parcae is a stable looped language model that matches the quality of a Transformer twice its size — a 770M model reaching 1.3B-level performance. We introduce the first scaling laws for looping and show that increasing recurrence, not just data, is a compute-efficient path to bet
- EinsteinArena: Harnessing the collective intelligence of agents in the wild to advance sciencetogether.ai
EinsteinArena is a platform where AI agents collaborate and compete on open math problems. AI agents on EinsteinArena have already set 11 new state-of-the-art results on open math problems — including pushing the kissing number lower bound in dimension 11 from 593 to 604.
New research shows LLMs can optimize database query execution plans—achieving up to 4.78x speedups by correcting the cardinality estimation errors that statistical heuristics miss.
- Inside the Together AI kernels teamtogether.ai
The team behind FlashAttention and ThunderKittens — how Together AI's kernel researchers close the gap between GPU hardware and production AI.
- Auroratogether.ai
1.25x over a well-trained static speculator. Aurora is an open-source RL framework that turns speculative decoding from a one-time offline setup into a self-improving system that learns from every request it serves.
As context windows grow, LLM performance degrades in unexpected ways. We show how a "Divide & Conquer" framework — breaking long documents into parallel chunks with a planner, workers, and manager — lets smaller models like Llama-3-70B and Qwen-72B outperform GPT-4o single-shot.
Together's fine-tuning service now handles tool-calling, reasoning and vision-language model formats natively, trains models above 100B parameters, claims up to 6× higher throughput, and surfaces cost and ETA estimates before a job starts.
- Mamba-3together.ai
Meet Mamba-3: the SSM built for inference. Faster than Transformers at decode, stronger than Mamba-2, and open-source from day one.
- FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingtogether.ai
As GPU throughput outpaces memory bandwidth, kernels must evolve. We introduce FlashAttention-4, featuring new pipelining for maximum overlap, 2-CTA MMA modes to reduce shared memory traffic, and a hardware-software hybrid approach to softmax exponentials.
- Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM servingtogether.ai
Serving long prompts doesn't have to mean slow responses. Learn how Together AI's CPD architecture separates warm and cold inference workloads to deliver 40% higher throughput and dramatically lower time-to-first-token for long-context LLM serving.
State-of-the-art speech models like Whisper and Deepgram score near-human on benchmarks — then fail 39% of the time on street names. New research from Together AI exposes the gap and a fix.
- Consistency diffusion language models: Up to 14x faster inference without sacrificing qualitytogether.ai
Standard diffusion language models can't use KV caching and need too many refinement steps to be practical. CDLM fixes both with a post-training recipe that enables exact block-wise KV caching and trajectory-consistent step reduction — delivering up to 14.5x latency improvements
What do language models generate when you don't tell them what to generate? New research reveals that LLM families have distinct 'knowledge priors'—GPT models default to code and math, Llama favors narratives, DeepSeek generates religious content, and Qwen outputs exam questions.



