WhichLLM

Why it's ranked here

The tool addresses more than basic memory fit. Its recommendations account for benchmark quality, evidence confidence, model generation, quantization, context memory, runtime overhead, and estimated throughput. Hardware simulation, planning commands, and machine-readable output make those recommendations useful beyond a single terminal check. The estimates remain planning aids, however, rather than measurements from the user’s machine.

What's good

Memory estimates include weights, key-value cache, activations, framework overhead, context length, and mixture-of-experts behavior. Recommendations distinguish full GPU loading, partial offload, and CPU execution. Benchmark evidence carries source and confidence information instead of being treated uniformly. Users can reserve VRAM and RAM headroom, filter slow candidates, select task profiles, and export JSON or Markdown.

Tradeoffs

Default recommendations are intentionally ambitious, so near-limit VRAM fits and partial RAM offload may prove less comfortable than expected. Speed values are estimates with confidence ranges, not live performance tests. Results normally depend on Hugging Face network data, though curated fallbacks cover offline or rate-limited cases. Ollama integration still requires users to map Hugging Face identifiers to local model names.

How to use it well

Use it when selecting a local model, checking whether a quantized variant fits, or comparing GPUs before purchase. Start with full-GPU filtering, a usable-speed threshold, and reserved VRAM when reliability matters more than ambition. Feed JSON into scripts for repeatable selection. Do not treat its throughput projections as a substitute for benchmarking the chosen model under your actual prompts and runtime.

Observed

License

MIT

Primary language

Python

Python requirement

Python 3.11 or newer

Packaging

PyPI package built with Hatchling; installable through uv, pip, or Homebrew

Interface

Command-line application with terminal tables, JSON, and Markdown output

Platform support

Detects NVIDIA, AMD, Intel, Apple Silicon, and CPU-only systems

Runtime formats

Supports GGUF through llama-cpp-python, AWQ and GPTQ through Transformers integrations, and FP16 or BF16 through Transformers