Latent Space: The AI Engineer Podcast
Why AI Compute Is a Rectangle — Erik Bernhardsson, CEO of Modal
Why AI Compute Is a Rectangle — Erik Bernhardsson, CEO of Modal
Elastic GPUs, the Rise of Sandboxes, and the Economics of the Agent Cloud
Transcript
A warm welcome to Basil Chatha, our newest co-host hailing from the AI engineering realm at Forward Deployed! We’re teaming up with him to launch a new show, and fresh off the heels of Modal’s massive Series D round, we couldn’t pass up an exclusive interview with Modal CEO Erik!
Two years ago, we talked with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, without forecasting workloads, and without paying for idle resources. Later, CTO Akshat joined the conversation to dive into the technical stack and infrastructure primitives Modal built to power these capabilities and agent experiences.
The past five months have seen monumental shifts. Since its $5B valuation Series C, Modal announced that its ARR has crossed $1 billion, raised a $750 million Series D at a $16 billion valuation, and has spun up nearly 3 billion sandboxes. In this conversation, Erik sits down with Vibhu to unpack the infrastructure and unit economics behind this explosive growth—and why real-world compute demand rarely matches the neat shapes drawn up in procurement contracts.
Watch the full interview on YouTube →
GPU Demand Isn’t a Rectangle
A conventional capacity contract typically commits a customer to a fixed number of GPUs over a fixed term. Plot capacity against time, and you get a rectangle. But real-world workloads look nothing like that: they ramp up, spike, and drop off throughout the day. If you buy enough capacity to handle the peaks, you pay for idle silicon; buy less, and you risk failing to serve demand.
Erik frames Modal’s breakthrough as a form of duration transformation: pooling demand across thousands of customers, locking in capacity under the hood, and letting each user consume it on demand as needed. The smoother the aggregated demand, the higher the utilization of that shared pool. This serves as a perfect companion piece to our conversation with SF Compute’s Evan Conrad: Evan is focused on making compute tradeable; Erik is focused on slicing it into smaller, more elastic pieces for consumption.
The flip side, however, is financial risk. If a provider locks in capacity at today’s rates and GPU prices later crater, it can end up saddled with expensive supply. Erik notes that while he doesn’t anticipate an imminent crash, it’s a possibility you have to account for when building the business. Jump to the GPU pricing discussion →
The Sandbox Product That Almost Got Cut
Modal launched its Sandbox offering after Erik and co-founder Akshat Bubna realized language models could generate code—and would inevitably need an environment to run it. Early adoption was lackluster, and the team even weighed deprecating the feature. Then came coding agents, background agents, and reinforcement learning, driving a massive wave of new demand. Today, Erik notes that sandboxes account for roughly a third of Modal’s revenue—an outcome he never saw coming at launch.
This surge has also reshaped the demands placed on the infrastructure itself. An agent might spin up an environment, execute code, inspect the output, and iterate through multiple attempts. Startup and resume latencies therefore become an integral part of the user experience. In our recent interview with Akshat, we explored this paradigm shift from Developer Experience (DX) to Agent Experience (AX); here, Erik provides the product and business context behind it. Jump to the sandbox discussion →
Fast Cold Starts Are an Infrastructure Feature
Elasticity only matters if capacity can come online fast enough. Modal’s deep dive on serverless GPUs breaks down the mechanics behind it: warm buffer pools of ready GPUs, a custom filesystem, process checkpointing, and GPU memory snapshots. Together, these building blocks minimize the overhead of bringing a fresh model replica live. It’s a prime example of how low-level infrastructure choices directly govern application responsiveness and cost.
We also covered Erik’s journey from recommendation systems at Spotify to founding Modal, the company’s compute procurement playbook, alternatives to NVIDIA, forward-deployed engineers, and why code reviews are optional at Modal. Jump to the engineering culture discussion →
Show Notes
-
Modal — The AI infrastructure platform founded by Erik.
-
Erik’s Blog — His essays on infrastructure, markets, and startups.
-
Modal’s Series D Announcement — The company’s latest growth milestones and sandbox stats.
-
How Modal Achieved Truly Serverless GPUs — The cold-start and scaling tech powering elastic inference.
-
Modal’s Resource Solver — How it optimizes capacity allocation across fluctuating prices and availability.
-
Scaling to 1 Million Concurrent Sandboxes — Engineering deep dive on sandbox scalability.
-
Our First Interview with Erik — The original thesis on serverless infrastructure.
-
Our Interview with Akshat Bubna — Infrastructure and developer experience in the age of agents.
-
The Programmable Datacenter — Evan Conrad on GPU trading and physical delivery.
Timestamps
-
0:00 What Modal does
-
1:25 From Spotify to founding Modal
-
4:19 Starting before the generative AI boom
-
7:43 Early customers and fundraising
-
10:00 Built specifically for inference and autoscaling
-
13:21 Where Modal sources its GPU capacity
-
15:39 Renting vs. building infrastructure
-
19:45 The downside risks of falling GPU prices
-
24:18 Is AI demand infinite?
-
26:12 NVIDIA alternatives
-
28:25 What Modal can learn from neoclouds
-
30:26 GPUs as an asset class
-
31:55 Modal’s core primitives: Functions and Sandboxes


