Banteg on Why AI Capability Gains Haven't Triggered Instrumental Drives

banteg points to a 2008 paper, "The Basic AI Drives" by Steve Omohundro, which argued that sufficiently capable, goal-directed AI systems would tend to develop instrumental drives such as self-preservation, resource acquisition, and further self-improvement, regardless of what final goal they were given. His observation is that eighteen years later, model capability has risen sharply while those predicted drives have not shown up in practice, which he offers as an encouraging sign.

That is banteg's own read of the current landscape rather than a resolved test of Omohundro's argument. The paper's prediction concerns sufficiently autonomous, capable systems pursuing goals over time, and most deployed models today still operate on narrow tasks under close human oversight, conditions the original argument treated as a precondition for the drives to emerge rather than as evidence against them. Whether the absence of instrumental drives so far reflects a flawed prediction, or that models have not yet been deployed under the conditions the argument describes, is a question his post raises but does not settle.