AA-Omniscience regresses 10 points from Qwen3.7 Max (+14 to +4), reversing its…
- Source
- x.com
- Date
AA-Omniscience regresses 10 points from Qwen3.7 Max (+14 to +4), reversing its predecessor's abstention gains. Accuracy is effectively flat at ~31% while the hallucination rate rises from 23% to 40%, meaning Qwen3.8 Max attempts more questions it cannot answer rather than declining them
Context
Alibaba's Qwen3.8 Max became more willing to give wrong answers on a knowledge test without a meaningful increase in correct answers. Artificial Analysis' August 6 results show accuracy staying around 31% while its AA-Omniscience score fell from 14 to 4.
The benchmark rewards correct answers, penalizes wrong ones and does not penalize refusals. Its reported error measure rose from 23% to 40%; that denominator includes incorrect, partially correct and unattempted responses, rather than all questions. The decline therefore reflects worse handling of uncertainty: more unsupported answers without a comparable gain in knowledge.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
Checking sign-in…
Loading comments…