Vibeleaderboard
← All Intel
Intel / article

Heaps do lie: debugging a memory leak in vLLM.

Source
mistral.ai
Date

Context

A stable heap profile can miss growing process memory. In Mistral AI's investigation of vLLM, software for serving language models, ordinary allocation tracking missed memory requested through lower-level system calls. The team traced the growth to UCX, a communication library whose queue of regions awaiting cleanup kept expanding. Disabling its memory hooks stopped the leak in their setup. The engineering account shows why comparing heap measurements with total resident memory, then tracing the unexplained allocations, exposed a problem the first profilers could not see.

Recommended reads
Comments

Checking sign-in…

Loading comments…