• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Gives a dated data point on how much frontier model output quality has moved on a fixed, well-known benchmark task, useful for calibrating expectations about the pace of visible capability gains.

• articleThe Pelican Benchmark (SVG Pelican on a Bicycle)Simon Willison
• articleAre AI labs pelicanmaxxing?Simon Willison
• articleThe Pelican comparison grid for Astra is pretty interestingsimonwillison.net

Checking sign-in…

Loading comments…