ADR: Agentic AI Detection and Response
github.com/uber/adr- Category
- Cybersecurity
- Rank
- No. 899Tools index
- Pricing
- Open Source
- Platform
- cli
- Type
- TOOL
- Builder
- uber
- GitHub
- 1.6k stars
- Latest release
- sensor-v1.0.0
- Date
About
ADR is an open-source enterprise security system for AI agents that combines telemetry collection (Sensor), a 300+ task benchmark covering 133 MCP servers and 17 attack techniques (ADR-Bench), and a two-tier dual-agent threat detector. It is deployed in production at Uber to monitor and secure both employee-facing coding agents (Cursor, Claude Code, Codex) and customer-facing support agents, with an accompanying paper accepted to MLSys 2026.
What it does
ADR watches agent conversations and tool activity, converts records from several coding assistants into a common event model, and analyzes sessions for malicious behavior. A fast screening stage handles clear cases, while suspicious sessions receive deeper model-based reasoning. Its bundled scenarios let security teams exercise this pipeline against realistic tool environments before applying it to collected activity.
Why it's ranked here
The project connects collection, threat detection, and repeatable evaluation instead of offering an isolated guardrail. Production use at Uber strengthens its operational case, while deterministic tests and packed conversations make core evaluation accessible. The published benchmark result is deliberately conservative: no false positives, but only 28 of 42 attacks detected. That makes ADR credible and useful, not comprehensive.
What's good
The Sensor normalizes logs from seven named agent sources and exports JSON or JSONL, including incremental per-session output. Detection records confidence, latency, token use, cost, and confusion-matrix inputs. Packed conversations avoid rerunning live agents, while explicit scored and dropped counts expose incomplete runs. Both Sensor and deterministic Detection components have unit tests, and synthetic fixtures avoid production credentials or customer data.
Tradeoffs
The open release excludes both action prevention and the offline red-teaming engine. Production telemetry and deployment results are also absent. Published benchmark recall is 67 percent, so zero false alarms came with missed attacks. Full detection needs model credentials, Claude command-line authentication, quota, and API budget. Benchmark dependencies include intentionally pinned packages with known vulnerabilities and must run in an isolated environment. Detector exceptions can also produce a benign default result.
How to use it well
ADR suits security teams operating coding assistants or support agents that need normalized telemetry, session triage, and repeatable adversarial evaluation. Start with the packed benchmark, verify quota, pin the results directory, and inspect dropped-task counts before trusting metrics. Run attack fixtures only inside a container, virtual machine, or dedicated host. Treat ADR as observability and detection infrastructure, not as an enforcement layer, production dependency set, or complete red-team platform.
Technical notes+
Sensor/adr_sensor/cli.py exposes the adr-sensor CLI with source selection, JSON or JSONL output, history controls, and incremental session saving. Sensor/adr_sensor/observer.py coordinates parsers for Claude Code, Cursor, Claude Desktop, Cline, Warp, Codex, and opencode. Detection/main_detector.py runs detector tasks concurrently, validates benchmark layout, calculates metrics, and records scored versus dropped tasks. Detection/guardrail/base_detector.py defines BaseDetector and DetectionResult; its task wrapper catches detector exceptions and returns a non-malicious result with zero confidence. docs/REPRODUCIBILITY.md documents packed-conversation inflation, detector execution, tests, and figure generation. docs/OPEN_SOURCE_REVIEW.md requires isolated benchmark execution because reproducibility pins retain dependencies with known CVEs.
Observed
- License
- Apache License 2.0; vendored AgentDojo code uses the MIT License.
- Primary language
- Python.
- Packaging
- ADR Sensor is packaged as adr-sensor for PyPI; both components use uv environments.
- Interfaces
- Sensor provides a Python library and command-line interface; Detection provides benchmark and detector command-line scripts.
- Platform support
- Sensor targets macOS, Linux, and Windows; Claude Desktop collection is limited to macOS and Windows.
- Testing
- Sensor has parser, schema, utility, and observer tests; Detection has deterministic pack, metric, and parser tests.
- Open-source scope
- Sensor, benchmark, and detector are included; prevention and the offline red-teaming engine are excluded.
Read from README.md, docs/RELEASING.md, docs/REPRODUCIBILITY.md, docs/OPEN_SOURCE_REVIEW.md, docs/BASELINE_REPLICATION.md, Detection/main_detector.py, Detection/openai_config.py, Detection/main_benchmark.py, Detection/plot_paper_figures.py, Sensor/adr_sensor/cli.py, Sensor/adr_sensor/__init__.py, Sensor/adr_sensor/observer.py, Detection/guardrail/__init__.py, Detection/guardrail/base_detector.py.
What it can do
Collect telemetry data from AI agents such as coding assistants and support agents
AI agent activity/telemetry → Collected telemetry data
Detect security threats in AI agent behavior using a two-tier dual-agent detector
Agent telemetry/logs → Threat detection results
Run the dual-agent threat detector against the ADR-Bench benchmark tasks
ADR-Bench benchmark tasks → Detection evaluation results
Tags
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.