Kev

Why it's ranked here

The headline claim is measured, not asserted. On datasets Kev never trained on, the largest size reaches 0.851 development accuracy against 0.857 for Jev, the hosted model it imitates, and the authors say plainly this is not a controlled comparison. Every number traces to a committed report, test sets are read once per released model, and the release notes list known weaknesses by name. Apache 2.0 throughout, with checksummed checkpoints and pinned weight revisions. That level of evaluation discipline is rare for a one-person model project.

What's good

Answers are distributions, not labels. The quick start ticket mentions a return, a late delivery and a double charge, and the department probabilities split 0.47, 0.28 and 0.25 accordingly. The release removed a silent failure: a document over 65,536 tokens is now refused with an error naming the count, rather than cut. Each model card states a validated context, the longest document where contract accuracy stays within 3 points of the 8k result. Fine-tuning is documented with honest numbers, including that 400 records gave a gain inside the noise.

Tradeoffs

The three smaller sizes trained on documents of at most 7,552 tokens and are validated only to 8,192, even though the server accepts eight times that. On datasets no Kev saw, the smaller sizes sit 13 to 31 index points below Jev. Date arithmetic is weak below the largest size, at 0.35 to 0.725 against Jev's 0.95. The 0.8B model's tool-call routing accuracy fell to 0.133, below chance. The largest model is overconfident on long contracts. The package declares itself alpha, and the 9B and 27B Mac paths are expected but not measured.

How to use it well

Start with the 4B size on a 24 GB GPU or a 32 GB Apple Silicon Mac, keep documents under roughly 8,000 tokens, and freeze confidence thresholds on a labelled sample of your own traffic. It fits classification, routing, triage and policy checks where you want to send uncertain cases to review. If you already call Jev through the TypeSafe Python client, swap the base URL. Turn on the date preprocessor for deadline questions. It does not generate, summarise or chat, it is English only, and it is not for unreviewed decisions with legal or medical consequences.

Observed

License

Apache-2.0 (code, adapters and heads; Qwen base models also Apache-2.0)

Language

Python, requires 3.12 or 3.13

Packaging

uv project installed from a git clone, optional serve and mlx extras; weights on Hugging Face Hub

Interfaces

HTTP endpoint compatible with TypeSafe System One; TypeSafe Python SDK; benchmark CLI module

Platform support

CUDA and ROCm GPUs; Apple Silicon through MLX

Model sizes

0.8B, 4B and 9B LoRA adapters; 27B full weights

Declared maturity

Development Status 3 - Alpha classifier in the package manifest

Tests

pytest configured with a tests directory and a server marker for live-endpoint tests