The Programmable Data Center — Evan Conrad, SF Compute

In this Latent Space conversation recorded at AI Engineer NYC, SF Compute's Evan Conrad explains the company's shape with a hotel analogy: you set out to sublet a room, then the guest wants cleaning, room service and security, and eventually you are running the hotel. SF Compute started by trying to sublease a cluster and has spent the following years, he says, building the whole operating capacity of a cloud. The reason is what the company calls physical settlement — a term borrowed from commodity futures, where settling a contract means the oil or the soybeans are actually delivered rather than a cash difference being paid against an index. For compute that means site visits, designing the bill of materials and the cluster, supervising rack rollout, and operating the switches, power distribution and management layer around the nodes being sold. Notably, SF Compute does not own the hardware: Conrad says that if it did, it could manipulate the forward market, so independence is a constraint on the business model rather than a slogan.

The non-fungibility problem is the part the headline does not convey. Rather than waiting for GPU hours to become a standard grade the way light sweet crude is, SF Compute built an order book where orders carry constraints and a matching engine pairs buyers with sellers whose capacity satisfies them. Conrad's examples of what has to be expressed: whether a workload can be interrupted and how fast it can recover from storage, whether the capacity is live now (selling it means interrupting whatever is running) or starts a month out, whether clusters have a fast private link so a job can be preempted in one place and resume in another, and which hardware generation and configuration is required. He adds a further dimension SF Compute is thinking about but has not built: early H100 and GB200 deployments had problems such as firmware bugs, so a sophisticated buyer might pay more for units deployed later. The admitted cost of this approach is thinner, more fragmented liquidity and wider spreads; the compensation, he says, is arbitrage for anyone whose workload can run on many kinds of capacity. He describes the market today as not very mature, deep or liquid, with about 5% of the company's total capacity on the order book and a tracked measure of usefulness: a customer who spends $100 on capacity recovers about $25 on average by selling some of it back.

The pricing discussion is where the mechanism becomes concrete. Because a block of compute cannot be sold in the past, capacity approaching its start date has to clear at whatever price it can get — Conrad's comparison is milk near its expiry — so the curve tends to dip at the front, rise, and then sit lower further out as buyers pay a premium for shorter lead times. Batch and live inference price differently for the same reason: a batch job needs an hour's notice, or less, to be moved, while a customer serving live traffic under an SLA needs a month or more. He also flags a gap he finds hard to explain away: a B300 running inference can earn roughly $40 an hour at full utilization, or around $25 at about half, while B300s can be bought at roughly five to six dollars an hour, and he says one of those numbers has to change. His warning about cash-settled compute futures follows the same logic as his case for physical settlement — an index built on thin data that correlates poorly with the price of actually buying capacity is both manipulable, the concern behind the LIBOR comparison raised in the conversation, and a source of basis risk for anyone hedging against it. He thinks cash settlement is a good idea once it is derived from physical settlement.

The independence argument closes the loop on why this role fell to a startup. Conrad's view is that NVIDIA benefits from such a market existing but cannot build it, because traders would not accept a venue run by the dominant hardware supplier, and that the same applies to clouds and inference providers with large demand-side businesses — making the market means giving up the cloud deals and inference revenue, so no incumbent will. For milestones he cites several hundred million dollars of volume closed, more clusters on the way, and a feature he is particularly pleased with: customers can now subdivide their InfiniBand fabric on the fly instead of filing a request, which he agrees amounts to virtualizing the data center and making it programmable. That capability is what gives the episode its title, and it matters to the market rather than only to engineers, since redrawing cluster boundaries on demand is what lets capacity scale up and down as orders match. SF Compute's earlier Latent Space appearance, covered in SF Compute: Commoditizing Compute, is the prior conversation this one deliberately picks up from.