Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Them
Source
Aditi Patodiya
Author
Aditi Patodiya
Date
Terms in this piece · Glossary
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Why it matters
Testing 327 quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition → model artifacts finds five official Ollama-library conversions, four Qwen2.5-Coder-3B builds and one phi3.5-mini, solve zero benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → tasks while independent conversions of the same models work fine.
articleTwo Truths and A Lie? Benchmarking Off-the-Shelf LLMs for Requirements Quality Assessment: Performance, False Alarms, and MissesJannatul Shefa, Alejandro Salado, Paul Wach, Taylan G. Topcu
articleDo large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effectsEmad Alharbi
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li