Vibeleaderboard

Search

Searching the tools index
Filters

Type

Ready to use
Build with
Connect / Integrate
Extend
Operate
Reference

Topics

More topics

Source

Ranked under Top · All Time

  1. 810

    WhichLLM

    Benchmarks local LLMs on your specific hardware and ranks which models run best on it.

    Utilitygithub.com/andyyyy64/whichllm

    andyyyy64 · Developer Tools

  2. 177

    LM Studio

    Desktop app for running local LLMs on llama.cpp and MLX, served over OpenAI-compatible local endpoints.

    Applicationlmstudio.aiFreemium

    @lmstudio · AI Tools

  3. 11

    llama.cpp

    C/C++ library for running LLM inference locally on CPUs and GPUs, with quantization options to cut memory use.

    Library / Frameworkgithub.com/ggml-org/llama.cppOpen Source

    ggml-org · AI Tools

  4. 1991

    Llmfit

    Terminal tool that scores which local LLMs will actually run well on your CPU, GPU and RAM.

    Utilitygithub.com/alexsjones/llmfitOpen Source

    AI Tools

  5. 102

    AirLLM

    Layer-by-layer memory management lets a single 4GB GPU run inference on 70B-parameter models.

    Library / Frameworkgithub.com/lyogavin/airllm

    lyogavin · AI Tools

  6. 689

    LocalAI

    A self-hostable AI runtime that runs LLM, vision, voice, image, and video models on any hardware behind one API.

    https://github.com/mudler/LocalAIOpen Source

    Developer Tools

  7. 1543

    rllama

    Ruby FFI bindings for llama.cpp that run open models like GPT-OSS, Qwen, Gemma, and Llama 3 locally.

    Library / Frameworkhttps://github.com/docusealco/rllamaOpen Source

    docusealco · AI Tools

  8. 324

    omlx

    LLM inference server built for Apple Silicon, featuring continuous batching and SSD caching.

    Runtimegithub.com/jundot/omlx

    jundot · AI Tools

  9. 251

    vLLM

    An open-source LLM inference engine using PagedAttention and continuous batching to maximize GPU throughput.

    https://github.com/vllm-project/vllmOpen Source

    Developer Tools

  10. 729

    llamafile

    Packages an LLM's weights with llama.cpp into a single cross-platform executable that runs with no installation.

    Runtimehttps://github.com/mozilla-ai/llamafileOpen Source

    Developer Tools

  11. 1374

    p1

    Local-first code completion engine from the ggml team that runs small models for editor autocomplete.

    Utilityhttps://github.com/ggml-org/p1Open Source

    ggml-org · AI Tools

  12. 2667

    VernLLM

    Adds retries, fallback, rate limiting, and caching around LLM API calls without a gateway.

    vernllm.dev

    AI Tools

  13. 1921

    Midtry

    Reasoning harness that queries several LLM CLIs in parallel, each prompted from a different perspective.

    Utilityhttps://github.com/hmbown/midtryOpen Source

    Hmbown · AI Agents

  14. 192

    Llama

    Meta's open-weight language model family spanning 8B to 405B parameters for local and cloud inference.

    Modelllama.comOpen Source

    meta-llama · AI Tools

  15. 1120

    llm-bridge

    Universal adapter that translates LLM input formats between providers, with built-in observability and error handling.

    Library / Frameworkhttps://github.com/supermemoryai/llm-bridgeOpen Source

    supermemoryai · AI Tools

  16. 1678

    rvLLM Serverless

    Lightweight vLLM replacement for RunPod serverless environments that cold-starts in seconds instead of minutes.

    Runtimegithub.com/instructkr/rvllm-serverlessOpen Source

    instructkr · AI Tools

  17. 1172

    Self-Adaptive LLMs

    Framework that lets language models adapt to unseen tasks in real time by composing expert modules on the fly.

    Library / Frameworkhttps://github.com/sakanaai/self-adaptive-llmsOpen Source

    sakanaai · AI Tools

  18. 2797

    TrackLLM

    Continuously monitors LLM API endpoints for undisclosed output changes over time, using logprobs.

    www.trackllm.net

    Developer Tools

  19. 2409

    llmbridge

    OpenAI-compatible LLM gateway in C++ with sub-millisecond translation overhead between providers.

    github.com/kottos-ai/llmbridgeOpen Source

    Developer Tools

  20. 456

    llama2.c

    Runs Llama 2 inference in a single file of pure C, serving as a minimalist reference for how LLM inference works.

    Template / Starterhttps://github.com/karpathy/llama2.cOpen Source

    karpathy · AI Tools

  21. 289

    LLMLingua

    Compresses prompts and the KV cache up to 20x with minimal accuracy loss, speeding up inference.

    https://github.com/microsoft/llmlinguaOpen Source

    microsoft · AI Tools

  22. 848

    Podman Desktop AI Lab

    Podman Desktop extension for running local LLMs in containers, from model discovery to inference and chat.

    Extensionhttps://github.com/containers/podman-desktop-extension-ai-labOpen Source

    containers · AI Tools

  23. 627

    Ollama JavaScript SDK

    JavaScript library for running, managing, and chatting with locally served Ollama models from Node or the browser.

    Library / Frameworkhttps://github.com/ollama/ollama-jsOpen Source

    ollama · Developer Tools

  24. 884

    TreeQuest

    Tree-search library from Sakana AI with a flexible API for scaling LLM compute at inference time.

    Library / Frameworkhttps://github.com/sakanaai/treequestOpen Source

    sakanaai · AI Tools

  25. 181

    LiteRT-LM

    Google framework for running large language models on phones, wearables, and IoT devices with GPU and NPU support.

    Library / Frameworkgithub.com/google-ai-edge/litert-lmOpen Source

    google-ai-edge · AI Tools

  26. 1681

    PostHog for OpenClaw

    Plugin that wires PostHog's LLM analytics into OpenClaw.

    Extensionhttps://github.com/posthog/posthog-openclawOpen Source

    PostHog · Developer Tools

  27. 165

    TensorRT-LLM

    NVIDIA's Python API for defining large language models and running optimized inference on NVIDIA GPUs.

    Library / Frameworkhttps://github.com/nvidia/tensorrt-llmOpen Source

    nvidia · AI Tools

  28. 778

    TensorRT Edge LLM

    Lightweight C++ inference engine that runs LLMs and vision-language models on edge devices.

    Runtimehttps://github.com/nvidia/tensorrt-edge-llmOpen Source

    nvidia · AI Tools

  29. 653

    LLM Council

    Karpathy experiment where an ensemble of LLMs debates a question and arrives at a synthesized answer.

    https://github.com/karpathy/llm-councilOpen Source

    karpathy · AI Agents

  30. 626

    OpenJudge

    Evaluation framework that scores LLM quality with reward models and grader skills for RLHF and agent alignment.

    Library / Frameworkhttps://github.com/agentscope-ai/openjudgeOpen Source

    agentscope-ai · AI Tools