Heretic

Why it's ranked here

The interesting bit is not the ablation math, which is published research, but that the parameters are searched instead of hand-tuned. On a 12B instruction model the author reports 0.16 divergence from the original against 1.04 and 0.45 for two widely used manual abliterations, at the same three refusals in one hundred prompts. Those are author-run numbers on one GPU, and the documentation says so. The design still holds up: two competing objectives, a sampler that respects both, and no knowledge of transformer internals required at the console.

What's good

The edit lands as a low-rank adapter instead of overwritten weights, so trials reload fast and you can distribute the adapter alone. Model loading tries four tensor dtypes in order and falls back when one fails, which is what keeps older GPUs working. Search state is checkpointed per model, so an interrupted run resumes where it stopped. Scoring is a real plugin interface: any class meeting the contract becomes a search objective, and each plugin declares whether it is deterministic, which the tool uses to decide whether a run can be offered as bit-for-bit reproducible.

Tradeoffs

Refusal detection is a keyword match. The default list holds roughly thirty case-insensitive substrings including sorry, illegal and harmful, so a model that answers fully but uses the word illegal counts as refusing, while one that refuses in unusual wording counts as compliant. The divergence objective sees one hundred harmless prompts and one hundred tokens of response. Older reproduction files are rejected rather than migrated after the plugin rewrite. And you need a GPU: the reference timing is twenty to thirty minutes for a four billion parameter model on a single consumer card.

How to use it well

For people running local models who hit refusals on legitimate work, and for interpretability researchers who want residual geometry printed or projected layer by layer. It fits between downloading a model and serving it: run once, keep the adapter, apply it at load time. It does not fine-tune, does not add knowledge, and does not repair whatever capability it costs you. If you want a behaviour added rather than a refusal direction removed, look elsewhere. Judge the result with your own prompts, not with the counter it optimises.

Observed

License

AGPL-3.0-or-later, declared in pyproject.toml

Language

Python

Platform

Python 3.10 or newer, PyTorch, GPU expected

Install surface

pip package heretic-llm, plus an optional research extra

Interface

single command-line program; no library API advertised

Configuration

TOML file and CLI flags, validated by pydantic

Extensibility

scorer plugin interface, loaded by dotted import path or file path plus class name

Output

merged full model or standalone LoRA adapter

Memory option

4-bit quantization via bitsandbytes

Model coverage

dense, several mixture-of-experts and multimodal architectures; pure state-space models not supported