Phobos – a tiny kernel language that now runs LLMs on a 2019 GPU

It shows how far runtime-compiled kernels can go on older consumer GPUs, with honest comparisons to llama.cpp on 1-bit-class quantizations and MoE models. Useful for anyone running local inference on 8 GB cards.

• mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

• postIntroducing the next evolution in LLM text generation: the first fully-fledged…Cohere
• postLiquid AI's 230M model runs agentic tool calls on a Raspberry PiLiquid AI
• videoCan LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AIAI Engineer

Checking sign-in…

Loading comments…