Guardrail Erosion in Domain Fine-Tuned Large Language Models: A Paired Audit of Knowledge-Centric Abstention in Biomedical LLMs

Researchers from Boston Consulting Group's BCG X AI Science Institute and the University of Cambridge compared 25 publicly released biomedical fine-tunes of open models with the 11 base models they came from, and found that most became even less willing to hold back an answer when the question lacked the evidence needed to answer it. The models descend from Meta's Llama 2, 3 and 3.1 and Google's Gemma-3, and were adapted by methods ranging from continued pre-training to preference tuning and model merging. Earlier work showed that fine-tuning can wear down a model's refusals of harmful requests. This audit looks at a different kind of caution, which the authors call knowledge-centric abstention: declining to answer, or asking for more information, because the facts given cannot support a conclusion.

The main test, MedAbstain, takes multiple-choice questions from MedQA, a set of US medical licensing exam questions, removes a key piece of clinical evidence, and offers an option to abstain. Base models already took that option rarely, a median of 8.3% of the time, and after fine-tuning the median fell to 4.2%. Abstention dropped in 21 of 25 pairs, a median relative decrease of 45%, and the result held after the authors corrected for multiple tests and for the fact that several fine-tunes share a base model. The fine-tuned models did not get better at telling answerable questions from unanswerable ones to compensate. On questions a fine-tuned model now answered where its base had abstained, its accuracy was a median of about 15 percentage points lower than on questions both models answered. Confidence in the chosen answer rose, including on questions both models got wrong. A second benchmark, ClinDet-Bench, which asks whether a clinical scoring rule can be applied when some risk factors are hidden, pointed the same way but was too small, with 94 public scenarios, to be conclusive.

The effect was not uniform. Four pairs became more cautious, capability and caution moved independently on MedAbstain, and none of the factors the authors examined, including recipe type, model size or training strength, explained which models lost caution once multiple testing was accounted for. The audit is observational, so model family, training data and method are tangled together. Many of these models were probably trained on MedQA, which could affect how readily they answer as well as how accurately, and human reviewers judged abstention appropriate for only 38 of 49 sampled MedAbstain questions with evidence removed. The authors say the results support a benchmark audit, not claims about clinical safety.

What the paper, to appear at the Machine Learning for Health symposium and available on arXiv, does establish is that an accuracy score alone would have hidden all of this. A biomedical fine-tune can post better exam results while answering more of the cases it should flag as undecidable, and with more confidence. The authors' recommendation follows from that: test each adapted model against its own base on caution, calibration and accuracy before trusting it where declining to answer matters, because in this sample no type of fine-tuning predicted which models kept their caution.