← All IntelClip / AI AgentsGuard the action surface, not the input
From Designing Multi-User Agents for Group Chats and Wearables · ≈9:12
“Instead of guarding whatever the agent reads, we let the agent read everything and then design a guard which is deterministic. So it's fast. We don't have issues with respect to latency.”
AI Engineer
“So within Judith there is a component which tries to look at this data and tries to build a supervised finetuning model an SLM which is Lora fine-tuned to try and predict if there is something malignant in the data channel separate the data channel firstly from the instruction channel and only give out that information which is purely benign.”
AI Engineer
“So the ideal response here would be if the instruction is summarize this page for me and the website that we have crawled contains some prompt injection attack or any other form of attack to obscure that information and the model just performs the task as required.”
AI Engineer
“A very interesting caveat here is that a particular uh attack was spotted where instead of writing the text normally if you write it interspersed with dots like I do. LIi. K. The regax based approaches and most of the static approaches fail at that.”
AI Engineer
What’s in it
- Concrete architecture choice: input filtering gates everything and adds latency, so put a fast deterministic check where the agent takes actions (bash variables, exports, secret configs) and bucket findings into allow, approve, block.
Clip transcript
The the papers have observed that around 90% of the attacks have this. So if you want to think about safeguarding our multi- aent multi-group setting architectures from this probably the first level of defense has to come at the boundary. Instead of guarding whatever the agent reads, we let the agent read everything and then design a guard which is deterministic. So it's fast. We don't have issues with respect to latency. But design when it is taking the action. So for example, when it is reading bash variables or when it is exporting something or when it is reading secret variables with respect to configs etc. And then since we can design a deterministic system using traditional NLP or regax based approaches we can flag these out saying hey some of them are benign so let's allow them but some of them need approval etc. So this is the three class categorization that we can come up with as a naive approach. Now uh we all know that this approach is good but we can probably do slightly better which would be to say can we train a model on top of it as well. So within Judith there is a component which tries to look at this data and tries to build a supervised finetuning model an SLM which is Lora fine-tuned to try and predict if there is something malignant in the data channel separate the data channel firstly from the instruction channel and only give out that information which is purely benign. So the ideal response here would be if the instruction is summarize this page for me and the website that we have crawled contains some prompt injection attack or any other form of attack to obscure that information and the model just performs the task as required. So uh we have also benchmarked this on inspect agent. That's one of the benchmark uh data sets available here and it's it's the naive approach here is basically at 50% which is a coin flip and it improves on top of that. A very interesting caveat here is that a particular uh attack was spotted where instead of writing the text normally if you write it interspersed with dots like I do. LIi. K. The regax based approaches and most of the static approaches fail at that. Whereas a learned model uh would probably catch it which is where the attempt to build a learn model comes
Comments
Checking sign-in…
Loading comments…