Skip to content
Main Site News Console

Why AI Compute Is Rectangular — Modal CEO Erik Bernhardsson

· Latent Space Translated
播客深度访谈

Elastic GPUs, Sandbox Growth, and the Economics of the Agent Cloud

Image 3

Latent.Space

October 22 · 86 minutes

A warm welcome to Basil Chatha, our newest co-host covering all things Forward Deployed in AI engineering! We’re launching a new show with him soon, and fresh off Modal’s massive Series D announcement, we naturally couldn’t pass up the chance to interview CEO Erik Bernhardsson!


Two years ago, we sat down with Erik Bernhardsson to explore his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, forecasting demand, or paying for idle resources. CTO Akshat joined the pod shortly after to break down the underlying infrastructure primitives and stack Modal built to deliver those building blocks and agent experiences.

A lot has changed over the past five months. Following their $5B Series C, Modal announced that their annualized run rate (ARR) has crossed $1B, alongside a $750M Series D at a $16B valuation, with nearly 3 billion sandboxes created to date. In this conversation, Erik joins Vibhu to break down the infrastructure and economics powering this growth—and why compute demand rarely matches the contracts used to buy it.

Watch the full conversation on YouTube →

Video 1

GPU Demand Is Not Rectangular

A typical compute capacity contract grants a customer a fixed number of GPUs over a fixed period. Plot capacity against time, and you get a rectangle. But real-world workloads look nothing like that: they ramp up, spike, and drop off throughout the day. If you buy for peak demand, you’re paying for idle hardware; if you under-provision, you risk falling short when it counts.

Erik describes Modal’s core bet as a form of “duration transformation”: pooling demand from thousands of customers, locking down capacity under the hood, and letting each customer draw from it on demand. The smoother the aggregate demand, the more valuable the pool becomes. This pairs nicely with our conversation with SF Compute’s Evan Conrad: while Evan focuses on making compute tradeable, Erik focuses on slicing it into smaller, more elastic increments for consumption.

The flip side, however, is financial risk. If a provider locks in capacity at today’s prices and GPU prices subsequently drop, they could be stuck holding expensive inventory. Erik notes that while he doesn’t expect prices to plummet overnight, any viable business model must account for that downside. Jump to the GPU pricing discussion →

The Sandboxes That Almost Got Killed

Modal originally launched sandboxes because Erik and co-founder Akshat Bubna noticed LLMs could write code—and people would need an environment to run it. Early adoption was lackluster, and the team even discussed shutting the product down. Then coding agents, background agents, and reinforcement learning triggered an explosion in demand. Today, Erik says sandboxes account for roughly a third of Modal’s revenue—an outcome he never saw coming at launch.

That growth has also reshaped the role of infrastructure. An agent might spin up an environment, run code, inspect the output, and retry multiple times. The latency of booting and restoring these environments becomes an integral part of the overall experience. In our recent interview with Akshat, we explored this shift from “developer experience” to “agent experience”; here, Erik brings the product and commercial perspective. Jump to the sandbox discussion →

Fast Booting Is an Infrastructure Feature

Elasticity only matters if capacity can come online fast enough. Modal’s deep dive on truly serverless GPUs details the underlying mechanics: warm pools of available GPUs, custom file systems, process checkpointing, and GPU memory snapshots. Together, these building blocks minimize the work needed to spin up a new model replica. It’s a prime example of how infrastructure-level decisions translate directly into application latency and cost.

We also touch on Erik’s journey from Spotify recommendation systems to founding Modal, the company’s compute procurement playbook, alternatives to NVIDIA, forward-deployed engineers, and why code reviews are optional at Modal. Jump to the engineering culture discussion →

Episode References

#播客#深度访谈#AI智能体#4Allapi.com

Published by the 4All API team

Original link:https://latent.space/p/modal-d/

Need a mainstream LLM API? 4All API gives you one key to call OpenAI, Anthropic, Google Gemini, Qwen, DeepSeek, and dozens more — at official-pass-through pricing, with enterprise-grade reliability, integrated in 5 minutes.

Sign up for the 4All API console →