Vibeleaderboard
← All Intel
Intel / article

Distributed databases with Peter Mattis

Source
newsletter.pragmaticengineer.com
Author
Gergely Orosz
Date

Context

Peter Mattis, co-founder and CTO of Cockroach Labs, says AI has changed how he explores engineering ideas. In the written takeaways from his Pragmatic Engineer interview, he describes thinking of experiments that would normally have taken him a week, then firing several off at once. He compares it to being a college professor with a swarm of research assistants whose results come back quickly.

That account offers a specific meaning of productivity beyond writing more code: an experienced engineer can investigate more alternatives before deciding what to build. Mattis says the experience of flow still exists, but it is somewhat less intense than hands-on coding and involves managing more things cognitively. The example explains how he uses agents to expand exploration; it supplies no measured productivity multiplier. His account of the workflow.

Key takeaways · AI-distilled
  • Colossus, the successor to Google File System, replaced three full replicas with Reed-Solomon erasure coding, cutting storage overhead by a third while increasing redundancy, according to Peter Mattis, who pioneered it with the team.
  • CockroachDB keeps three replicas by default and up to five for some system tables: with one replica a crash is unrecoverable, and with two neither side can tell whether the other received the last write. More replicas are allowed but add latency.
  • Because Colossus was append-only, Spanner could not use B-trees, which need in-place updates, so LSM trees fit instead. CockroachDB began on RocksDB, an LSM engine forked from LevelDB, and now runs on Pebble, which Mattis wrote and open sourced in 2019.
  • Mattis replaced std::map, a red-black tree with two pointers per node, with a B-tree at Google that was faster through spatial locality and smaller. He later built a Swiss Table map for Go that the Go team helped land in the standard library.
  • A former Google C++ readability reviewer, Mattis says he reviews less code as agents improve and predicts engineers will stop looking at code the way they stopped reading assembly. Non-engineers at Cockroach Labs built about 1,000 internal apps in a couple of months.
Read the source newsletter.pragmaticengineer.com
More from Gergely Orosz
Recommended reads
Comments

Checking sign-in…

Loading comments…