The Watchdogs of AGI — Rune Kvist of AI Underwriting Company

Shows how standards, adversarial testing and insurance are emerging as the trust layer for deploying autonomous agents, which affects what enterprise buyers will require of agent products.

• AIUC announced a $40M Series A after a seed led by Nat Friedman and Dan Gross, and works with companies including Cursor, Harvey, Lovable and ElevenLabs.
• Kvist argues AI standards may need updating every quarter rather than every decade, and that standards and insurance need to evolve together; the episode covers how Lloyd's of London can insure AI systems.
• He says most AI companies optimize the happy path without seriously stress-testing adversarial cases, and that every model can ultimately be jailbroken.
• The Air Canada chatbot case is cited as an early example of AI failures clarifying legal liability, and copyright is described as possibly one of the hardest AI risks to insure.

• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• jailbreak — A prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.
• red teaming — Deliberately attacking your own AI system to find what makes it fail before someone else does.

• video🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics)
• video🔬 Google's AI Scientist Started as an Attempt to Automate Kaggle — John Platt, Google Fellow
• videoA Worm With 302 Neurons Inspired Their Architecture — Ramin Hasani, Liquid AI
• videoOrbs: Shifting Coding to Cloud — Quinn Slack, Amp Code

• articleUnderwriting Superintelligence: Backing Agents you can Sue — Rune Kvist, AIUCwww.latent.space
• videoThe Next Chapter of AI Is Inside Our Softwarea16z
• videoWhich AI startups actually land enterprise contracts? — Brian Lewis, MillenniumAI Engineer

Checking sign-in…

Loading comments…