A substitution test shows LLMs answer drug questions from name patterns

A model can sound like it knows a drug—even when it doesn’t. Researchers at @UTAustin, @Northeastern, & @UTMDAnderson found LLMs often lack drug-specific knowledge and instead lean on morphology. Because Olmo is open, they could use it to trace why. 🧵 https://t.co/sjFbKqoEFx

@UTAustin @Northeastern @UTMDAnderson Many drug names contain clues to their class: “-pril” → ACE inhibitors “-olol” → beta blockers “-azoline” → certain decongestants Those clues can let a model answer from the structure of the name rather than from knowledge about the specific medication.

@UTAustin @Northeastern @UTMDAnderson The researchers built a diagnostic to separate 3 signals models like Olmo might rely on: → the affix, like “-pril” → the drug’s unique stem → actual drug-specific knowledge They replaced parts of drug names with invented strings & measured how Olmo 3’s answers changed.

@UTAustin @Northeastern @UTMDAnderson The results showed a big gap between sounding knowledgeable & knowing the drug. For 51–59% of tested drugs, Olmo 3’s response to the real name was barely distinguishable from its response to a made-up one. Another 12–18% appeared driven by the affix itself.