[](https://huggingface.co/ErickvL)
- Utilization, not intelligence, is the next real constraint in AI.
- The bottleneck moved from models to compute
- Why busy clusters still waste capacity
- Intelligence moves into the infrastructure
- Specialization frees capacity; orchestration spends it
- Further reading
Utilization, not intelligence, is the next real constraint in AI.
The aviation industry learned this the hard way. For most of its history, the single metric that best predicted an airline’s survival was how much time each aircraft spent on the ground per day.
The reason is structural. An aircraft’s costs accumulate continuously by the calendar hour: financing, depreciation, hull insurance, scheduled maintenance, and crew contracts. Its revenue, however, only accumulates by the flight hour. Every hour spent on the ground squeezes the output on the right side of the equation while the costs on the left side run as usual. Utilization is also downstream of almost every other operational decision an airline makes. Turnaround discipline, route network design, maintenance scheduling, crew scheduling, and spare parts supply all ultimately manifest in this single number—because if the underlying operations break down, the plane stays on the ground, no matter how well everything else is executed.
A larger fleet, of course, still helps. More planes mean more available capacity, which is obvious. But two airlines operating fleets of similar sizes on similar routes can end up with vastly different economic performances, and most of that gap often stems from this single metric, not the size of their fleets.
Enterprise AI is running into the exact same structural reality on different hardware. GPUs also incur costs by the calendar hour—regardless of whether they are doing useful work at any given moment—driven by financing, depreciation, power, and cooling. Their output only accumulates by the compute hour. Having more GPUs helps in much the same way a larger fleet helps an airline: it certainly brings more capacity and is indeed an advantage, but it still doesn’t guarantee the bottom-line result. Two companies with similar GPU budgets are increasingly diverging based on how much of that hardware is actually doing useful work at any given moment, rather than how much hardware they each own. Just like airline utilization, this number is downstream of almost every other infrastructure decision an enterprise makes. Intelligence brought the industry to where it is today, but utilization is becoming the next real constraint.
The bottleneck moved from models to compute
As AI scales, scarcity hasn’t disappeared; it has simply moved up the chain to a different resource.
The first wave of enterprise AI victories belonged to model quality. Larger models, trained with more compute, and evaluated on more rigorous benchmarks: parameter scale and leaderboard rankings dominated the conversation, and this race did indeed produce models good enough to run real enterprise workloads. But this capability came with a dependency. Production AI runs on specialized hardware, which today is almost exclusively GPUs.
GPUs are expensive, supply-constrained, and demand far outstrips supply—a reality that holds true even at the very top of the market. In 2020, Microsoft built a dedicated supercomputer for OpenAI: over 10,000 GPUs and 285,000 CPU cores. At the time, the system was reported to be one of the top five in the world, built specifically to train the model that would become GPT-3. Back then, this seemed like an almost unimaginable concentration of hardware, enough to make it feel like the compute problem was solved as long as you could get your hands on it. Six years later, that number feels more like a starting point than a ceiling. By 2026, even the world’s best-capitalized labs view compute acquisition as an ongoing strategic constraint rather than a settled issue. Anthropic alone is advancing multiple gigawatt-scale commitments across four different hardware platforms simultaneously: Amazon, Google, Microsoft, and AMD, layering them on top of each other within months. Meanwhile, Meta has signed multi-gigawatt deals of similar scale. Placing bets on four suppliers at once is what compute scarcity looks like when the buyer has virtually unlimited capital but still cannot secure enough compute from any single source.
Separated by six years, both events mark the frontier a lab must reach to remain competitive. What has truly changed in the interim is not so much that AI has gotten stronger, but that capability itself is no longer the defining bottleneck.
The same pattern is playing out outside the labs, just in a different form. Enterprises consuming these models via APIs face pricing issues rather than hardware issues. Costs scale linearly with the number of tokens used, a factor that almost completely separates the economics of a proof-of-concept (PoC) from those of a production environment. A PoC handling a few thousand requests a month looks affordable; scaling that same workload to production can turn costs into an expense that never truly converges. The alternative increasingly adopted by enterprises is straightforward: buy their own GPUs, run models on-premise, and swap variable, linearly scaling costs for fixed capital costs.
API costs rise with usage, while the cost of self-owned infrastructure remains nearly fixed. Beyond the break-even point, this trade-off reverses.
This shift turns GPUs from an expense item into infrastructure: provisioned for growth and peak demand, meaning they are provisioned at a scale higher than what is actually needed in any given week. In other words, the purchase doesn’t solve the problem; it opens up a new one. The day the cluster goes online, the question is no longer can we get accelerators? but rather can we keep them busy? Yet, initially, only the first question had a procurement team assigned to it. Signing the hardware contract is the part with a deadline and an owner. Keeping it from sitting idle is the part that quietly decides whether the deal was worth signing in the first place.
These deals describe capacity commitments, not efficiency. How well that capacity is actually utilized is a different question; it is managed by different people, measured far less rigorously, and is much further from being solved.
Why busy clusters still waste capacity
A cluster fully loaded with busy GPUs can still waste most of its potential, and the reason is almost always the same. GPUs run around the clock, but the demand placed on them does not. Infrastructure must be provisioned for the peak: the moment when training tasks, batch jobs, and real-time traffic all arrive simultaneously. This means that outside of peak times, a significant amount of capacity is reserved but left unused. If every GPU could handle all types of workloads equally well, better forecasting could solve this problem on its own. But in reality, very few GPUs can, which is precisely the other, harder half of the problem.
This mismatch starts at a deeper