why did everyone suddenly decide specialized inference engines are a good idea?
i've seen at least 5 of these released in the last month.
what's wrong with vLLM/sgLang?
sure you can generate a slop inference engine quickly now but why fragment the ecosystem? it's harmful.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
It surfaces a real debate for anyone building inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → infrastructure: whether the recent proliferation of narrow, AI-generated inference engines is healthy competition or wasteful duplication of vLLM/sgLang's work.
postLocal inference crosses over: the M5 Ultra and the 1,000x claim
