Enterprise AI Buyers Want a Cloud, Not Just GPUs

Here's the thing: the early GPU boom assumed the market would be a short list of hyperscalers and frontier labs placing enormous orders. Enterprises building AI-native applications are changing that picture. Their needs look less like bulk accelerator purchases and more like a working cloud that can run distributed, secure workloads across regions.

Capacity alone stopped being the bottleneck

The first wave of so-called neoclouds built large GPU clusters wherever they could secure land, power and shell space, then sold that capacity to a small number of very large buyers. That model made sense when the customer was a single hyperscaler. It makes less sense when the customer is a software team that has to deploy, monitor and secure inference across several sites.

Kevin Cochrane, chief marketing officer of The Constant Company, which does business as Vultr, describes the shift in blunt terms. "In the early market … it was all about just standing up large GPU clusters wherever you could get land, power and shell, and then just selling it on through to a hyperscaler," he said. "And somehow in that mix, people forgot that you still need cloud, right? You still gotta run this thing."

Vultr is betting that the winners in this phase will be full-stack clouds rather than pure capacity providers. The company has committed to AMD silicon and open software stacks as the foundation of that strategy.

Why enterprise buyers change the architecture

Cochrane frames the current moment as one stage in a longer cycle. Each computing supercycle, in his telling, reshapes the stack from the data center up to the application layer. The first AI iteration was tuned for a narrow set of very large buyers, and that concentration pulled the market back toward centralized systems.

"We took a little bit of a step backwards because in that early iteration of the new architecture, when we over-indexed on a limited number of buyers, we went back more to centralized systems," Cochrane said. "And we have to remember that the long arc of compute is towards decentralized distributed systems driven by open standards and driven by open source."

For platform teams already running Kubernetes, that argument has practical weight. Many organizations are moving from cloud-native stacks to AI-native ones, and they want the orchestration, observability and security tooling they already know to extend to GPU workloads. A provider that can't run containers, and increasingly virtual machines, alongside accelerators forces a second platform into the company.

Inference, KV cache and the networking problem

Distributed AI also changes how inference gets served. Many users send similar prompts, and the key-value cache built during one request can be reused to skip recomputation for another. Vultr says it wants to maximize that token reuse with a global namespace that shares KV cache across nodes. The company is pursuing that work with partners including Vast Data, Cochrane noted.

The idea is simple to state and hard to build. A shared cache only helps if nodes can reach each other quickly and reliably, which pushes the problem into the network. Cochrane argues that connectivity between data centers matters as much as raw compute, because enterprises want private networks that are isolated from every other tenant on a public cloud.

"In the neocloud moment, I think some of the complexity of what's required to operate workloads at global scale for enterprises has been forgotten or just assumed it wouldn't matter," Cochrane said. "But … it's the biggest barrier to getting enterprises to actually leverage GPUs from a new cloud because the networking connectivity, it just doesn't work."

That's the part I'd watch most closely. Teams evaluating a new GPU provider often focus on the accelerator generation and the hourly rate, but cross-site latency, tenant isolation and cache behavior under real traffic tend to decide whether a deployment works at all. The interview didn't include benchmarks for any of these, so they remain claims until buyers test them.

Open source as part of the infrastructure bill

Vultr is also putting resources behind the open-source projects that make up the AI stack. The company has joined the Cloud Native Computing Foundation as a platinum member. At KubeCon it highlighted its Open Source Credits Program, which provides infrastructure to maintainers behind new CNCF projects.

Cochrane's point is that new projects can't mature without test and build capacity, and maintainers rarely have budgets for it. "All of those projects need compute, right? The maintainers aren't going to suddenly open up their own personal checkboxes," he said. "How can you build the next great CNCF project if you don't have compute?"

Credits programs like this are common across cloud providers, and their effect depends on how many projects they reach and how long the support lasts. Still, the underlying problem is real. If the open-source layer depends on donated compute, the enterprises building on it are indirectly exposed to that supply.

What to watch next

My read is that the next phase of GPU competition will be decided less by who signs the largest single contract and more by who can make multi-site, multi-tenant AI behave like ordinary cloud infrastructure. Watch whether cross-region networking and shared caching become standard offerings, and whether Kubernetes tooling absorbs GPU scheduling cleanly enough that platform teams don't need a separate stack.

A disclosure matters here too. Vultr sponsored this segment of theCUBE, and the company's positioning is the lens for the whole conversation. The architectural argument is worth taking seriously, but it should be tested against independent performance data for distributed inference before it shapes a buying decision.