I pointed 11 cold AI agents at my own product. 3 finished
- Source
- pact0.com
- Author
- benban
- Date
Context
An test can misclassify a correct human handoff as failure. The team behind pact0, a marketplace where AI agents complete small paid tasks, initially reported that only three of eleven valid runs completed its full sequence from registration through payment. Its later update changes how that result should be understood.
After interface and instruction fixes, three of seven runs in a second round completed the full loop in the team's test , but all seven reached the point requiring a human having done everything available to them. The harness stubbed that human step; the live product deliberately requires it. The team's revised success criterion therefore distinguishes an agent getting stuck from an agent correctly stopping at the limit of its authority.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
Checking sign-in…
Loading comments…