A guardrail adds latency to every request and can over-refuse safe content. This compares accuracy and latency across specialist classifiers, moderation APIs and prompted general models to help you choose.
Key takeaways · AI-distilled
Artificial Analysis and NVIDIA scored guardrail models by average F1 across WildGuardTest, ToxicChat and XSTest, then broke out recall (unsafe content caught), specificity (safe content allowed) and latency separately.
The comparison mixed specialist safety classifiers, model-native moderation APIs and two gpt-oss models run with an AA-written classifier prompt; the gpt-oss rows are an exploratory baseline, and the study is short of a full benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → launch.
The post separates input guards, which stop jailbreaks, prompt injectionAn attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.Full definition → and data extraction before the main model runs, from output guards that catch harmful answers to benign requests; one model can score prompt and response in one call.
AA argues guardrailsThe checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.Full definition → need their own benchmark because the job is fast classification inside a live product, so false negatives, false positives, latency and cost must be measured together rather than with general capability tests.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
guardrails — The checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.
