Refusing the Impossible: A Taxonomy and Benchmark for Code Hallucination in Large Language Models
Source
Vishnu Asutosh Dasu, Ashish Kundu, Gang Tan
Author
Vishnu Asutosh Dasu, Ashish Kundu, Gang Tan
Date
Terms in this piece · Glossary
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Why it matters
Gives engineers a taxonomy and 270-prompt benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → to test whether coding models correctly refuse impossible or ungrounded tasks instead of confidently fabricating plausible-looking code.
articleFrom Detection to Refusal: Safer LLMs via Circuit-Guided Weight ScalingKuan-Lin Chu, Chung-En Sun, Tsui-Wei Weng
articleCan a Model Catch Its Own Hallucinations for Free?: Label-Free Doubt Signals Hold Their Own Against a Labelled Dataset for AbstentionAli Asaria, Tony Salomone, Deep Gandhi
articleDo Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code GenerationAlex Deaconu, Anubhav Gupta, Manaal Basha, Nicholas Haydu, Gema Rodr\'iguez-P\'erez