Extending LLDB to Debug Applications That Span CPUs and GPUs

Meta engineers have built a way for LLDB, the debugger from the open-source LLVM compiler project, to follow a program as it moves between ordinary CPU code and code running on a GPU, and most of the shared framework is already in LLVM. Their engineering post starts from how AI software runs: a workload can begin in a high-level framework and move back and forth between native CPU code and kernels, small programs that run on the GPU. A bug that crosses those boundaries needs a debugger that understands both sides. Historically that has meant a vendor's own tool, such as AMD's ROCgdb or NVIDIA's CUDA-GDB, each of which has to build and maintain common debugger features separately.

The difficulty is that a debugger normally assumes one process with one processor architecture and one consistent set of threads, registers and memory. A GPU breaks each assumption. The host may run on x86-64 while the kernels use a different instruction set; the GPU runs groups of lanes in lockstep, sometimes thousands of them, which do not map onto the thread lists debuggers display; and the CPU and GPU can have separate address spaces, so the same address can point to different memory on each. Squeezing all of that into one artificial process would make every operation, from reading memory to stepping, first ask which hardware it is about.

Meta's answer is to keep the two apart and coordinate them. LLDB already uses a helper process, lldb-server, that actually controls the program being debugged and talks to the LLDB client over the GDB remote protocol. The new plugin interface lives there. When a program uses a supported GPU runtime, the plugin asks for internal breakpoints at points in the host program such as the moment a GPU debugging session can begin. When the program reaches one, the plugin starts the vendor's device-debugging interface and opens a second connection, and LLDB creates a second target for the GPU with its own architecture, threads, registers and memory. The two targets are technically independent but appear in one LLDB or IDE session. In the post's sample session, a breakpoint on a GPU function waits until the GPU code loads, then stops there and shows a backtrace through the kernel. Programs that never touch a GPU keep the existing path with no added runtime overhead, and a mock plugin lets LLVM's test suite check discovery, breakpoint coordination and connection setup without GPU hardware. The same framework handles GPU core dumps after a crash.

The case for separate targets was discussed in an LLVM forum thread begun in February 2026, whose opening post says Meta and NVIDIA had been working together on branches with AMD and NVIDIA support. LLDB developer Jonas Devlieghere, one of the upstream reviewers Meta thanks, replied that separate targets for the CPU and the accelerator were "the right call." The thread's author then noted that CUDA-GDB and ROCgdb use a single target and stop the CPU whenever the GPU stops, which, at least with NVIDIA, can occasionally deadlock; another participant questioned whether that limit comes from the single-target design or from GDB's default stopping mode. In July an Intel engineer posted an Intel GPU plugin built on the same plugin model to the fork where this work was being developed. Meta says the plugin interface, the new protocol messages, multiple address-space support and the mock tests are upstream, with the remaining common pieces expected soon. The vendor plugins themselves are not described as upstreamed, so the post does not establish when developers can use this with a released LLDB on their own hardware. The stated goal is that supporting new hardware should require only a vendor plugin, not another separate debugger.