Purple Llama

Why it's ranked here

The project covers an unusually broad security workflow, from standards-based evaluation to concrete runtime decisions. Code inspection returns issues, severity, and block-or-warn guidance rather than a bare verdict. The package is less cohesive than that scope suggests: onboarding points to separate repositories, and licensing differs between benchmarks, models, and safeguards.

What's good

The code inspector supports eight common languages and combines regular-expression rules with Semgrep analysis. It can scan every supported language concurrently when callers do not specify one. Findings carry CWE identifiers, severity, location, rule details, and suggested treatment. The benchmark suite spans prompt injection, code-interpreter abuse, spear phishing, vulnerability exploitation, and autonomous offensive operations.

Tradeoffs

Code inspection catches every analysis exception, logs it, and returns the same non-insecure, ignore result used when nothing is found. Callers therefore need separate failure monitoring to avoid treating scanner outages as clean code. Semgrep must be available through its package or the system path, and setup creates a nearby symbolic link. Model safeguards also use Llama Community licenses rather than MIT.

How to use it well

Use it when an LLM team needs repeatable adversarial evaluation before release and runtime gates around prompts, responses, or generated code. Record benchmark results, then connect warning and blocking decisions to application policy and operational monitoring. It does not supply a complete deployment layer: the project directs adopters to separate Llama reference-system and recipe repositories for integration guidance.

Observed

Licensing

Cybersecurity evaluations and Code Shield use MIT; model safeguards use the corresponding Llama Community licenses.

Interfaces

Code Shield provides an asynchronous Python library interface; ClassifyIt provides a console command.

Packaging

ClassifyIt uses setuptools, requires Python 3.8 or newer, and declares Llama Stack, YAML, Tika, Peewee, and Google API dependencies.

Code analysis

Code Shield uses regular-expression and Semgrep analyzers and reports CWE-linked issues with severity and optional source locations.

Supported scan languages

Active Code Shield scanning covers C, C++, C#, Java, JavaScript, PHP, Python, and Rust.

Benchmark structure

The documentation sidebar lists ten cybersecurity benchmark areas, including prompt injection, spear phishing, autopatching, and threat-intelligence reasoning.