Intel / article

Using Local Coding Agents

Source: magazine.sebastianraschka.com
Author: Sebastian Raschka, PhD
Date: Published Jun 27, 2026

Context

A local coding needs both a language model and software that lets it work on a project. Sebastian Raschka's tutorial separates these roles: a local model server generates responses, while the reads files, applies edits, runs commands and checks results.

That separation explains why downloading a capable model is only part of the setup. Raschka checks generation speed and memory use as inputs grow, then tests coding and tool-selection behavior. A model that chats comfortably may still struggle with the longer inputs or precise actions needed for useful coding work.

Key takeaways · AI-distilled

A local stack splits in two: an server (Ollama, vLLM, LM Studio, MLX) that serves the weights, and a harness that gives the model file reads, edits, shell commands, and verification. Either side swaps independently.
Models get tuned against their maker's own harness. Nvidia's Polar paper benchmarked Qwen3.5-4B across coding harnesses and it scored best inside Qwen-Code, both before and after RL training, so pairing model to native harness is free accuracy.
Qwen3.6 35B-A3B is a roughly 22 GB download that wants 30-40 GB of RAM, and it runs at usable speed on a Mac Mini M4 or a DGX Spark. The floor for a local coding agent is a well-specced desktop, not a GPU cluster.
On Apple Silicon, pull the *-mlx build of a model: it is compiled against Apple's Metal shaders. Linux boxes take the plain tag. Same weights, different runtime, meaningfully different speed.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

Key quotes

“As we can see, Claude Code uses by far the most tokens on average, Codex the least.”

“For example, one Claude run used about 578k input tokens but only about 4.5k output tokens across 25 turns.”

Read the source magazine.sebastianraschka.com