Vibeleaderboard
← All Intel
Intel / article

NaiveAI Promises 2,100 Tokens per Second. How Well Does It Code?

Source
latent.space
Author
Gabriele Beltramo
Date

Context

Latent Space tested Naive-N0.5-Flash, a new open-weights coding model from the Chinese company NaiveAI, and found that its headline speed is real for typical questions but not a promise for every request. NaiveAI built the model by further training Xiaomi's MiMo-V2.5, adapting it to DeepSeek Sparse with about 3.25 trillion more tokens, and serves it on its own NaiveRT engine. In Latent Space's runs against DeepSeek V4.1 Flash 1017 from DeepSeek and GLM 5.3 Flash from Z.ai, on 30 questions ranging from easy math to very hard reasoning, the median question took 3.1 seconds with Naive's thinking mode, against 5.3 seconds for GLM and 10.1 for DeepSeek at their high thinking settings. The averages were much closer, 45.3 seconds for Naive and 45.0 for DeepSeek, with GLM at 86.9. On the hardest questions Naive was slower on average than DeepSeek, 89.8 seconds against 79.7, and it wrote about 108,500 tokens per answer there, roughly 11 times DeepSeek's average.

The 2,100 tokens per second in the title is a peak over a single second, which the review says to treat as an upper bound. The speed comes from running on NaiveAI's own inference engine: a small draft model guesses several tokens ahead and the main model checks them in one pass, so when the guesses are right several tokens come out for the cost of one pass of the main model. NaiveAI reports about 984 tokens per second while the model is thinking and 1,816 while it writes the answer, measured on one eight-GPU Blackwell server tuned for a single user's latency. Guesses were accepted 70.6% of the time with thinking off but only 43.4% with it on, which helps explain why the model is slower while thinking. Latent Space's throughput numbers for Naive came from NaiveAI's own servers while the others came from OpenRouter, so it declines to state a speed multiple, and it did not grade whether any answers were correct.

In three app-building tests, Naive with thinking on built the most detailed solar-system app, tied DeepSeek on a 3D tree generator and came second to DeepSeek on a browser ray tracer, while GLM was weakest throughout. With thinking off, Naive failed to produce a usable tree app. NaiveAI's target price of $0.40 per million output tokens sits in the same band as the two rivals, but its long reasoning traces mean the bill per hard question can be higher. Latent Space's conclusion is that the model suits developers who value fast iteration over token economy.

Key takeaways · AI-distilled
  • Naive-N0.5-Flash starts from Xiaomi's MiMo-V2.5 (310B parameters, 15B active) and was adapted to DeepSeek Sparse Attention with about 3.25T additional training tokens, roughly 3T of them at a fixed 1M , according to NaiveAI.
  • In Latent Space's 30-question test, Naive-N0.5-Flash with thinking had a 3.1 s median response against 5.3 s for GLM 5.3 Flash and 10.1 s for DeepSeek V4.1 Flash 1017 on high, but its 45.3 s mean matched DeepSeek's 45.0 s. Answers were not graded for accuracy.
  • Thinking makes the model verbose: on hard HLE questions Naive-N0.5-Flash averaged about 108,500 completion tokens, roughly 9x GLM 5.3 Flash and 11x DeepSeek V4.1 Flash 1017, while its median across all 30 questions was about 3,200 tokens.
  • The 2,100 tokens/s figure is a one-second peak from speculative decoding on one eight-GPU Blackwell node. Reported decode is about 984 tokens/s while thinking and 1,816 while answering, as draft acceptance drops from 70.6% to 43.4% with thinking on.
  • In the app tests, Naive-N0.5-Flash with thinking built the most detailed solar-system app and a correct ray tracer, but with thinking off it produced no usable 3D-tree app. DeepSeek V4.1 Flash 1017's ray tracer was the most efficient of the three.
Terms in this piece · Glossary
  • attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
  • inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
  • speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Read the source latent.space
Recommended reads
Comments

Checking sign-in…

Loading comments…