Recent thoughts:
The Shift to Long-Horizon Tasks
The most likely breakthrough this year will be in long-horizon tasks. We are moving toward a stage where Large Language Models (LLMs) learn to complete extended, complex missions by interacting with Agent environments. This is perhaps where the true value of LLMs lies. Take cybersecurity as an example: imagine a model that continuously hunts for software bugs and vulnerabilities. While it sounds like a search process, it’s actually the model learning the high-level intuition and methodology of a professional hacker. Unlike humans, AI can run 24/7 without fatigue. It could potentially find exploits at a much higher frequwill ency and claim bounties on platforms like HackerOne or BugCrowd. It sounds fun, but fundamentally, it's a revolution that displaces the hacker. If even hackers are being "disrupted," one can only imagine the impact on general programmers.
From One-Person to None-Person Companies
Building on long-horizon capabilities, Autonomous Agent Systems (AAS) will inevitably become the next frontier. Last year, we were discussing the rise of the "One Person Company" (OPC). I didn't expect us to move so quickly toward the…
Tang frames near-term cybersecurity progress as a model learning a professional hacker's methodology rather than brute-force search — able to run 24/7 and claim bounties on platforms like HackerOne and BugCrowd.
For each pillar, Tang gives a concrete marker: memory via 1M+ context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → and RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition →, continual learning via release cycles shrinking toward weekly, and self-judging via early self-correction already visible in models like Opus 4.7.
Tang speculates top labs may already be self-training models — having them write their own code, clean their own data, and generate then train on synthetic data — and ties Anthropic's rumored 2-million-chip cluster to exactly this use.
Tang argues AGI should be redefined as the sum of collective human intelligence rather than any individual's, with the bar set at producing insights as significant as the Theory of Relativity.
Tang predicts the traditional desktop and app model gives way to an 'LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → OS' that generates applications on demand, calling it a challenge to the 80-year-old Von Neumann architecture.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Jie Tang lays out memory, continual learning, and self-judging as the three pillars needed for agents to run long autonomous missions, framing long-horizon task capability as the likely next AI breakthrough.
Read the full post on X ↗
