- Category
- Cybersecurity
- Rank
- No. 241Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- nvidia
- GitHub
- 9.1k stars
- Latest release
- v0.16.0
- Date
About
NVIDIA's LLM vulnerability scanner — probes models for jailbreaks, prompt injection, data leaks, hallucinations, and toxicity at scale.
What it does
Garak sends selected attack prompts to a model, then applies matching detectors to the responses. It can test local models, hosted services, or REST endpoints. Runs show progress, mark undesirable behavior as failures, report failure rates, and save detailed records for later analysis.
Why it's ranked here
Garak is a credible choice for repeatable model failure testing because it combines static, dynamic, and adaptive probes with detector-based evaluation. Broad provider support, configurable probe selection, parallel execution, detailed logs, and confidence interval controls make it useful beyond one-off prompt experiments.
What's good
The plugin structure separates model interfaces, probes, response detectors, and prompt transformations. Users can scan broadly or target a family, individual test, taxonomy tag, or tier. Seed control aids repeatability, parallel request and attempt settings accelerate remote scans, and each run preserves detailed JSON records.
Tradeoffs
The default behavior tries every known probe, which can produce many generations and potentially substantial hosted-model usage. Several providers require credentials, REST targets need configuration, and recognized OpenAI model types are whitelisted. Installation also brings a large Python dependency set, including model, cloud, language, and media packages.
How to use it well
Use Garak for security engineers, model evaluators, and developers who need repeatable adversarial checks before deployment or after model changes. Start with narrowly selected probes, fixed seeds, and controlled generation counts, then preserve reports for comparison. Treat its findings as assessment inputs: the documented workflow detects and reports failures, but does not remediate model behavior.
Technical notes+
pyproject.toml defines a Python package built with Flit, requires Python 3.10 or newer, and exposes garak through garak.__main__:main. garak/__main__.py forwards command-line arguments into garak/cli.py, whose parser supports generator selection, unified probe and buff specifications, detector overrides, parallelism, configuration files, interactive mode, and report processing. garak/data/__init__.py restricts resource lookup to configured data roots and rejects traversal outside them. garak-report/src/App.tsx and garak-report/src/main.tsx show a separate React report interface with persistent light, dark, and system themes. docs/source/conf.py configures Sphinx API documentation and extensive redirects for plugin modules.
Observed
- License
- Apache-2.0
- Primary language
- Python
- Installation
- Published as a pip-installable package, with source and editable installation documented
- Interface
- Command-line tool with interactive probing and report-processing modes
- Python support
- Requires Python 3.10 or newer; package classifiers list Python 3.10, 3.11, and 3.12
- Model access
- Supports local models, multiple hosted providers, and configurable REST endpoints
- Packaging
- Uses Flit as the build backend
- Platform metadata
- Classified as operating-system independent, with repository test workflows shown for Linux, Windows, and macOS
Read from README.md, pyproject.toml, requirements.txt, docs/README.md, garak/cli.py, garak/__init__.py, garak/__main__.py, docs/source/conf.py, garak/data/__init__.py, garak/buffs/__init__.py, garak-report/src/App.tsx, garak/probes/__init__.py, garak-report/src/main.tsx, garak/analyze/__init__.py, garak/intents/__init__.py.
What it can do
Probe LLM for jailbreak vulnerabilities
Large language model → Jailbreak vulnerability assessment report
Test model for prompt injection attacks
LLM and test prompts → Prompt injection vulnerability results
Scan for potential data leaks
Language model → Data leak detection report
Detect model hallucinations
LLM responses and test queries → Hallucination detection analysis
Analyze model outputs for toxicity
Language model → Toxicity assessment report
Perform large-scale vulnerability testing
Multiple LLMs and test scenarios → Comprehensive security vulnerability report
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.