Vibeleaderboard
← All Intel
Intel / article

A Governance-Aware Large Language Model Orchestrated Agentic Digital Twin for Transmission System Operator Control Room Decision Support

Source
arxiv.org
Author
Costas Mylonas, Magda Foti, Emmanouel Varvarigos
Date

Context

A prototype power-grid assistant places enforceable software controls between a and the analysis tools it can use. The model selects tools, while a separate governance layer restricts execution, requires operator approval for actions with side effects and inserts numerical results from the backend into answers. In the authors' study, all four governance rules held across 590 primary runs on a digital model of the Greek transmission network. The tests assume the governance layer itself is trusted, and the authors call formal verification of it a continuation.

Correctly sourced numbers can still answer the wrong question. The paper records 11 tool-selection misses in the primary campaign, including answers drawn from the wrong data source or the wrong day, even though the controls held. The prototype demonstrates containment under tested conditions; whether it improves control-room operators' decisions remains a planned .

Key takeaways · AI-distilled
  • The system is evaluated on a released 118-task benchmark built on a digital twin of the Greek transmission network, covering analytics, simulation, multi-step workflows and twelve families of adversarial inputs.
  • Across 590 runs of the primary model, tool selection reached 96.5% and task success 93.7%, and all four governance rules held in every run, checked against a persistent audit trail.
  • The rules also held in all 1,416 runs of a separate three-repetition study across four LLMs, which the authors report as an approximate 95% lower bound of 99.8%.
  • With the governance layer removed, the same model executed all 45 approval-requiring runs without authorization and only 39.2% of its answers had backend-supported numbers; enforcement costs 12 to 16 ms per request.
Terms in this piece · Glossary
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Recommended reads
Comments

Checking sign-in…

Loading comments…