MosaicLeaks: Can your research agent keep a secret?
- Source
- huggingface.co
- Date
Context
A research can reveal confidential information through a sequence of web searches even when no individual query contains the whole secret. ServiceNow researchers' MosaicLeaks study tests whether an observer who sees only the accumulated queries of an agent that combines internal documents with public information can reconstruct private facts.
In the reported Qwen3-4B experiment, training for task success alone raised the share of research chains answered correctly at every step from 48.7% to 59.3%, but answer-or-full-information leakage rose from 34.0% to 51.7%. Adding a privacy penalty cut that leakage to 9.9% while the success share stayed at 58.7%. The penalty considers both the new query and what it reveals alongside earlier searches. These results come from synthetic enterprise documents, a fixed web collection and one answering multi-step questions, so they show a promising training method within that controlled setting, not leakage rates in deployed systems. Experiment and limitations.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…