Intel / video

Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium

Source: youtube.com
Author: AI Engineer
Date: Published Oct 12, 2026

Context

Separating tenants' GPU memory can still leave clues about their activity. In a published experiment, security company Stealthium reports that certain GPU-wide memory operations on an NVIDIA H100 divided into Multi-Instance GPU partitions altered the timing of reads inside a neighboring tenant's partition. Measuring those delays revealed patterns associated with the neighboring model's text generation, even though the probe did not read its memory.

The talk's published description distinguishes identifying activity from recovering prompts or answers. The supported concern is that isolated memory contents can coexist with observable timing patterns in the tested hardware configuration; the attacker also has to share the same physical GPU.

Key takeaways · AI-distilled

Brzak's H100 attack probes only the attacker's own memory and exploits no software bug: hardware operations that affect the whole GPU leak timing across memory and compute isolation.
From the timing trace he infers model depth and decoding speed, then workload categories such as code, prose, JSON and retrieval. He states it does not recover prompt or answer contents.
Decoding speed alone is a weak fingerprint because many models decode at similar rates; layer signatures help separate related architectures, and workload in his setup depends on 's acceptance patterns.
On an NVSwitch-connected H100 server he shows a weaker but still informative signal across NVLink. He frames these as results from tested configurations, and suggests watching for probing workloads on the GPU itself.

Terms in this piece · Glossary

inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.

Read the source www.youtube.com