DeepSeek V3 in pure JAX

About

A 190-line JAX implementation of the DeepSeek V3 architecture, including Multi-head Latent Attention, DeepSeekMoE with auxiliary-loss-free balancing, and multi-token prediction. It was trained from scratch on public-domain detective fiction on a MacBook Air. The code is a readable reference for how these techniques work, and the author states it is an architecture demo rather than a strong language model.