The Watchdogs of AGI — Rune Kvist of AI Underwriting Company
- Source
- youtube.com
- Author
- Latent Space
- Date
Context
AIUC, an AI underwriting company, argues that the biggest limit on AI adoption may be trust rather than capability, because companies need to know who is responsible when an autonomous system fails. In the Latent Space interview, cofounder Rune Kvist describes AIUC-1, a standard for security, safety, and reliability, and says agents are stress-tested for jailbreaks, hallucinations (confidently wrong answers), and data leakage. Episode description
The company's proposal ties standards, testing, and insurance together: the standard sets what to examine, the tests look for failures, and insurance, the interview says, can put a financial backstop behind deployment, with Lloyd's of London named as a possible insurer. The intended benefit is a clearer basis for accepting deployment risk than a successful demo. That is AIUC's argument, and the description does not show what any policy covers or how well the tests work. Episode description
- AIUC announced a $40M Series A after a seed led by Nat Friedman and Dan Gross, and works with companies including Cursor, Harvey, Lovable and ElevenLabs.
- Kvist argues AI standards may need updating every quarter rather than every decade, and that standards and insurance need to evolve together; the episode covers how Lloyd's of London can insure AI systems.
- He says most AI companies optimize the happy path without seriously stress-testing adversarial cases, and that every model can ultimately be jailbroken.
- The Air Canada chatbot case is cited as an early example of AI failures clarifying legal liability, and copyright is described as possibly one of the hardest AI risks to insure.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- jailbreak — A prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.
Checking sign-in…
Loading comments…