Manthan Gupta
2 Apps · 1 Intel
Manthan Gupta is an AI research engineer based in Bangalore, India, focused on the infrastructure powering modern AI — inference engineering, GPU kernels, and distributed training. He works as an independent AI engineer consultant and builds across agents, LLMs, and distributed systems, guided by a philosophy of "build to understand, write to clarify."
Is this you? Sign in with X to claim this profile.
Apps
Production-ready agent harness framework for Python — build multi-agent workflows with first-class observability.
An autonomous AI agent that automatically optimizes LLM inference speed on Apple Silicon by running experiments overnight. It hill-climbs on tokens per second by modifying inference code and uses git commits as experiment tracking.
Intel
Most "AI optimization" demos are fun to watch for the same reason benchmark tweets are fun to watch: they show you the win, not the search. You see the final graph. You see the +12% or the "runs 2x faster now" claim. What you usually do not see is the graveyard of bad ideas behind it. The settings that looked promising but were just noise. The optimizations that made throughput better by quietly making the model worse. The fake wins that only happened because the benchmark got easier. So I built a small repo called Auto-Inference-Optimiser (star the repository!) to study exactly that. The idea is simple: lock the evaluation, open one file for experimentation, and let an AI coding agent hill-climb on inference speed forever on Apple Silicon. The most interesting part was not that it achieved a speedup. It was the kind of speedup it achieved, what it failed to improve, and what that indicates about inference engineering on real hardware. Let's get into it. Why I Built This I care a lot about inference right now. Not in the abstract "LLMs are cool" sense. I mean the actual production questions: where latency comes from, what batching buys you, what prompt processing costs, how KV cache decisions change throughput, and where the hardware wall starts pushing back. There is a lot of content online about training. There is also a lot of content online about agents. But there is still not enough content that combines both instincts: build a tight experimental harness, let the agent search inside it, and use that process to learn something real about inference. This repo was my way of doing that. It is inspired by Karpathy's Autoresearch, but pointed at a different layer of the stack. Instead of searching over training code on a GPU box, this one searches over an MLX inference pipeline on a Mac because I am GPU poor (please sponsor a GPU). What The Repo Actually Does At a high level, the repo turns "make inference faster" into a bounded optimization problem. The structure is