Enki LLM

About

Enki LLM is a Transformer inference engine of about 2,000 lines of Rust that runs Llama-3.2 1B without CUDA, reporting about 40 tokens per second on a Tesla T4. Its kernels are Rust functions compiled to SPIR-V and run through Vulkan 1.3, with a CPU fallback using Rayon. Slash commands move generation and the KV cache between GPU and CPU mid-response, and a Colab notebook is provided to try it.