Substrate

Why it's ranked here

The case rests on a real cost problem and a serious build. Idle pods still hold memory, and the Kubernetes scheduler needs seconds to start one, which the design document argues is too slow for work lasting milliseconds. Substrate answers with snapshot and restore onto warm capacity: Apache 2.0, written in Go, with a published threat model and fail-closed credential handling. Two outside projects, Agent Executor and the CNCF Sandbox project kagent, are listed as building on it. Against that, it is pre-1.0 with no compatibility promise, and its own architecture page says much of the design is not yet implemented.

What's good

Secrets handling is the standout. An agent calls an outside API with a placeholder in its authorization header, and an outbound gateway swaps in the real token, so the secret never sits in the agent's memory, files or snapshots. Each failure case is spelled out and fails closed: a missing secret returns 403, an unreachable secret provider returns 503, and plain HTTP is never given a credential. Secret access starts as default-deny. Snapshots come in two depths, full memory plus files for a hot resume, or data only for a cheap one. Internal components authenticate each other with short-lived certificates over mutual TLS.

Tradeoffs

This is early infrastructure. The README calls the project pre-1.0 and very young and warns that APIs may change significantly. The team's own threat model says it has little to no security hardening yet. Fine-grained authorization on the API sits behind an experimental flag that is off by default. The architecture page lists autoscaling of warm pods and protection against losing locally held state as unsolved. Its headline targets, 100ms wakeups at the 95th percentile and a billion agents per cluster, are goals, not measurements. Running it means operating PostgreSQL, object storage and several custom components. A gVisor-based agent gets only one persistent data folder.

How to use it well

The fit is a platform team already on Kubernetes that hosts many stateful agents, coding environments or tool servers that sit idle most of the day and need memory and files preserved between bursts. Start with the local kind quickstart and the counter demo before touching a cloud account; the Google Cloud path creates IAM grants and needs beta Kubernetes features switched on at cluster creation. Grant secret access per atespace deliberately, since the default grants nothing and edits need a provider restart. It is not a framework for writing agents, and teams that need a stable API today should wait.

Observed

License

Apache 2.0 (LICENSE)

Language

Go (go.mod declares go 1.27.0)

Platform

Kubernetes; aims to support the latest stable release and the previous minor (README.md)

Interfaces

kubectl plugin CLI and a gRPC API served by the ateapi binary (docs/glossary.md)

Sandbox runtimes

gVisor and micro-VMs (docs/glossary.md)

Snapshot storage

GCS or S3 object storage (docs/glossary.md)

State store

PostgreSQL (docs/threat-model.md)

Maturity

Pre-1.0, no backward-compatibility guarantee (README.md)

Credential handling

Egress gateway injects secrets and fails closed (docs/egress-credential-injection.md)