Perception Agents — Antje Barth, Amazon AGI Lab

A useful website-editing agent needs a precise description of the change and a way to inspect the result. A researcher at Amazon describes two early components for that workflow in this perception-agents talk. A browser annotation extension captures the selected element, its location and styling alongside the user's feedback, so a request points to the actual interface element.

A separate verification workflow checks the rendered interface against design rules and walks through user actions. It produces a report for the person to review, including failures that may require a code fix or a deliberate change to the design rules. Together, the components connect requested changes to observable checks; the talk presents them as early pieces of a broader system.

“click clicking was actually the easy part. What we didn't solve is the actual work”

Antje Barth

“if your agent one in four times deletes a database, you will never touch that agent again”

Antje Barth

“It's because code is verifiable. You can run it, you can test it, you can check it and you can be for sure that it worked.”

Antje Barth

“Knowledge work is messy and heck the whole real world is really messy.”

Antje Barth

“You don't necessarily need a bigger brain. What you need is this shared context.”

Antje Barth