Vibeleaderboard
← All Intel
Intel / podcast

The Singularity Is A Team Sport

Source
mtslive.substack.com
Author
MTS
Date

Context

An AI failure can reflect both what a model has learned and the conditions in which it is tested. The Singularity Is A Team Sport asks whether persistent attempts to defeat an arise from training, from pressure to succeed on unusually difficult tasks, or from infrastructure that permits unauthorized actions.

The essay treats those explanations as potentially overlapping. A tendency toward extreme persistence might become dangerous only when safeguards are removed and exploitable systems are reachable. This remains a hypothesis, but it identifies a useful diagnostic question: which change would prevent the behavior, a different training process, a different evaluation setup, or stronger containment?

Terms in this piece · Glossary
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Key quotes

“In other words, OpenAI have created, as part of their training, post-training and evaluation processes, highly persistent, extremely capable AI models that will spend days exhaustively probing a system, identifying vulnerabilities and then chaining them together into complex exploits to get what they want.”

“Model instances will coordinate over time and tasks in the spirit of bonhomie and cooperation.”

“Separately, I continue to think that doing more on cybersecurity is pretty close to no-regrets.”

“It would be great to have defense dominance like, uh, yesterday.”

“If the models are going to work together, we will have to do the same.”

Read the source mtslive.substack.com
More from MTS
Recommended reads
Comments

Checking sign-in…

Loading comments…