Muse Spark 1.2 continues Meta's AA-Omniscience pattern: the score rose from 18…
- Source
- x.com
- Date
Muse Spark 1.2 continues Meta's AA-Omniscience pattern: the score rose from 18 to 22, driven by abstention for the second consecutive release. The hallucination rate fell 10 points (38% to 28%) as the attempt rate dropped to 67%, while accuracy slipped from 41% to 38%. Heavy abstention drives both the low hallucination rate and the lower accuracy
Context
Meta's Muse Spark 1.2 improved its AA-Omniscience score while answering fewer questions correctly overall. Artificial Analysis reported a score increase from 18 to 22, alongside an accuracy decline from 41% to 38% and an attempt rate of 67%.
The benchmark's scoring rules explain how those results can coexist: correct answers earn credit, incorrect answers lose credit, and abstentions receive neither. Answering fewer uncertain questions can therefore improve the score even when fewer questions receive correct answers. The reported improvement concerns willingness to avoid guessing, rather than increased factual coverage.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
Checking sign-in…
Loading comments…