Intel / article

Choosing Where and How to Moderate: End-to-End Trade-offs in Filter Placement and Response Rewriting

Source: arxiv.org
Author: Mengya Hu, Susie Park, Suzana Ilic, Qiong Wei, Sandeep Atluri, Myra Deng, Tucker Fross, Curt Tigges
Date: Published Jul 30, 2026

Context

Where a moderation filter intervenes changes both the useful answers people receive and the harmful material they see. This study compared filtering requests, responses or both. At the tested settings, response-only filtering delivered the highest share of shown answers that were relevant and non-harmful, while filtering both sides reduced harmful exposure further. Rewriting flagged responses recovered much of the blocked traffic, but some rewrites omitted potentially useful safety information. Equal observed harmful-response counts in one comparison did not establish equivalent safety. The findings support evaluating the entire user-visible outcome when choosing a moderation setup. Study abstract.

Key takeaways · AI-distilled

Judge moderation on two end-to-end numbers, not classifier accuracy: Usefulness (share of turns showing a helpful, non-harmful reply) and Harmful Exposure (share showing a harmful one). Latency and errors are diagnostics, not the score.
Placement is a real tradeoff with no winner. Filtering only the model's response gave the highest usefulness; filtering both the user input and the response gave the lowest harmful exposure.
Rewriting a flagged response instead of blocking it recovered most of the blocked traffic, at the same measured harmful count as blocking. The rewrite generalizes the triggering language while keeping the benign intent.
Routing with a small probe classifier cut route-and-generate time substantially versus asking an to route, at comparable outcomes. The routing decision does not need a big model.
Rewrites have a quiet cost: in sensitive domains, the softened answer sometimes drops safety-relevant support information the user actually needed.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Key quotes

“Content-moderation classifiers are usually evaluated in isolation, but deployment requires choosing where to intervene and what follows a flag.”

“At the evaluated operating points, Response only achieves the highest filter-only Usefulness in both settings, while Input + response achieves lower Harmful Exposure.”

“Replacing Response only blocking with Response + rewrite recovers most blocked traffic and yields the same observed Harmful Exposure count as Response only blocking for the selected configuration; this equality is not an equivalence result.”

“These results support comparing moderation configurations under deployment-specific safety and latency constraints rather than applying a universal placement rule.”

Read the source arxiv.org