Vibeleaderboard
← All Intel
Intel / article

Time Capsule of Testable Human Knowledge: 41 Years of Jeopardy! in a Single Free Local Model

Source
arxiv.org
Author
David Noever, Forrest McKee
Date

Context

A locally served Qwen2.5-14B model answered 67% of 529,939 Jeopardy! clues under the authors' scoring procedure. The study used a roughly 9 GB, four-bit model served locally through Ollama and required an answer to every clue, scoring responses through normalized exact or approximate text matching.

The result demonstrates substantial question-answering capacity in a portable model, but its meaning depends on that setup. The full sweep ran on a single workstation-class GPU, and the authors treat memorization of older clues from training data as probably true to some degree, without quantifying it. The headline percentage measures performance on the collected questions; the paper does not evaluate a live game with selective buzzing.

Recommended reads
Comments

Checking sign-in…

Loading comments…