llama.cpp

About

A C/C++ library for running large language models locally with minimal setup and optimized performance across different hardware architectures. Enables LLM inference on CPUs and GPUs with various quantization options to reduce memory usage.