Local Agentic Theory For Mobile Games — Shafik Quoraishee & Joanne Song, The New York Times

An on-device AI agent could adjust a game's controls when the interface gets in a player's way. In an experimental accessibility talk, New York Times engineers describe simulations in which an agent detects a keyboard-focus loop and adds an escape route, or identifies undersized controls and enlarges them. These are research examples; the speakers explicitly say the work is not a feature of the newspaper's games.

Running those adjustments locally could avoid waiting for a server response, but the agent must share the phone's memory, processing time and battery with the game. The proposal's value depends on improving access without making play stutter. The speakers identify faster models and better testing as work still needed.

“Our puzzles are made by people. They're not made by AI.”

Shafik Quoraishee

“RL changes the model itself and changes the weights in the model”

Shafik Quoraishee

“you get a new frame every 16 ms a pound if your refresh rate is 60 Hz. You need to actually fit the agent planning within that”

Shafik Quoraishee

“The agent rewrites the layout live adapting to the human and not the other way around.”

Joanne Song

“the future of AI doesn't have to be one giant centralized brain. It can be billions of small local brains, each running on a personal device, each shaped entirely by the individual it serves”

Joanne Song