Intel / article

Does Graph Structure Earn Its Place in Microservice Root-Cause Analysis? A Controlled Study on RCAEval, and What the Benchmark Was Really Measuring

Source: arxiv.org
Author: Imad Bulji\'c
Date: Published Sep 24, 2026

Context

A system can look good at locating software failures by learning which components are usually blamed. A study of the RCAEval benchmark reports that faults were injected into only five services per system, although monitoring data covered 12 to 70. A ranker using no monitoring data still placed the true culprit among its first five guesses in 99.7% of held-out incidents.

That shortcut changes how results should be interpreted. The authors also found no reliable benefit specifically attributable to graph structure when comparing otherwise matched models. High performance within this can therefore reflect knowledge of its restricted culprit pool, making it difficult to tell how much a model learned from the actual failure symptoms.

Key takeaways · AI-distilled

In-distribution, the graph model led an otherwise identical flat model by only 0.003 Avg@5 (p = 0.844 over six disjoint folds).
RCAEval injects faults into only five services per system while exposing 12 to 70 in telemetry, so a ranker reading no telemetry puts the culprit in the top five on 99.7% of held-out incidents (Avg@5 0.488).
The author argues that telemetry-free prior, not the uniform-random 0.137, is the honest in-distribution floor; it falls to 0.192 across systems.
A non-uniform column schema silently zeroes telemetry for most RE1 cases, a second benchmark property that conditions any result on it.
The proposed PSC-GRCA model reaches mean Avg@5 0.915 versus 0.864 for the flat baseline, but its ablations place most of the gain in the prior term, not the graph.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.

Read the source arxiv.org