Vibeleaderboard
Index / tool
Visit github.com
Category
AI Agents
Rank
No. 2323Tools index

Previous survey · No. 2158 ·

Pricing
Open Source
Type
TOOL
Use case
Software Testing & Quality
GitHub
2 stars
Date

About

Untyped model-checks the protocol around an AI agent harness against a TLA+ spec, both in the design and in recorded runs. Because runtimes retry, a tool call can be dropped, delivered twice or run with a lost acknowledgement, and the check tests whether the harness still gives at most one effect per step, no destructive effect without approval and no more calls than budgeted.

At a glance

Untyped uses a TLA+ model to check how an agent harness handles retries, approvals and execution budgets. It is worth reading if a lost acknowledgment could cause a tool call with side effects to run twice.

Good fit

Agent runtime developers who can map their own protocol or traces onto a small formal model.

Consider something else when

Use integration tests first if you need evidence about a specific deployed system and have not modeled its behavior.

Start here

Read the duplicate-delivery example and compare the correct and buggy models. Then map one recorded workflow to the assumptions and see which events the model can represent.

Cost and requirements

MIT-licensed. The documented checks need Java 11 or later and the repository’s model-checking workflow.

Limits to know

The result covers the modeled states and assumptions. Recorded traces do not establish that unobserved execution paths behave the same way.

Why it made the leaderboard

Untyped lets you model-check whether your agent harness actually enforces at-most-once effects, required approvals, and call budgets under retries and dropped acknowledgements, verified against both the spec and real recorded runs.

Tags

tla+model-checkingai-agentsinvariantsverificationretries

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.