The birth of generative AI spawned a new generation of compute providers, mostly leveraging access to Nvidia GPUs to offer neocloud services faster and cheaper than the hyperscale giants.

Companies like CoreWeave, Nebius, and Iren rode this wave to great success and are now major cloud providers in their own right, facing the same challenges of maintaining a huge data center portfolio.

With agentic AI now pitched as the next evolution of the generative era, new chip providers and presumptive cloud companies are hoping to once again reshape the market, focusing on low-latency inference close to the end customer.

This feature was published earlier this year in the Agentic AI supplement. Read it for free today.

One such effort is Vector Core Compute (VC2), a curious combination of private equity and novel compute that hopes to prove that the Edge has finally found its killer app in agentic AI. Backed by Vista Equity Partners and Cambium Capital, and using SambaNova hardware, we sat down with the company for its first exclusive interview.

VC2 has committed to spending some $3.5 billion over three years on SambaNova, a compute provider that designs its own Reconfigurable Data Unit (RDU) AI chips, which it packages in a broader platform that includes Intel hardware.

Both Vista and Cambium were already SambaNova investors, but say they saw an opportunity in moving further down the chain and providing the compute to customers - some of which they have also invested in.

"Cambium is an early-stage VC that invests in companies that advance compute, so we do mostly hardware companies - we did like Groq, Rivos, and Samba, and things like that," Cambium co-founder and managing partner Landon Downs explains.

"We think these investments are great for data centers, and given the way the world is moving, we decided we should actually start building data centers ourselves."

At the same time, Vista was seeing a shift in its investments, which were historically primarily in the software space. Its portfolio businesses were increasingly spending growing sums on agentic AI, or developing their own agentic AI.

"By verticalizing the use cases for the enterprise all the way down through the hardware stack. you end up with a superior solution in our mind because you intentionally purpose-build the stack so we can meet the enterprise, and that was kind of the genesis of it," Downs says.

While much of the market has shifted to chase ever larger deployments, Downs says that his experience with a property in Canada taught him that "you don't actually end up with economies of scale necessarily when you're building large-scale mega projects.

"In some cases, you actually get significantly better economics doing these smaller-scale projects; so we're focusing on low-cost, low-latency, high-performance data centers."

Downs believes that, with the right hardware close enough to city centers, “you can have the compute near where you actually want to consume it. The data centers will be inference-heavy, but also think of it as refinement and training capabilities, plus Edge storage, so you'd be able to execute low-latency inference where you want it. You also could collect and store data and do refinement training all in one place, and you could do this at economics that work better for enterprise customers.”

The result of these concurrent beliefs is a plan to spread across up to 50 metropolitan centers in the US, rapidly vaulting VC2 into the upper echelons of North American Edge providers.

"These will be 5-20MW data centers, and help create this ultra-low latency fabric," Downs says. “We're finding sites that meet a set of criteria, entering into a lease for the space, and then we're working with the owner to put in the upgrades that we need to meet the data center specs. For some of them, they could be prior data center sites, and for other ones, they could be not at all structured as data centers, but buildings that have the right specs.”

Some ten sites are "very close to ready to go," with a facility in Los Angeles already live.

"LA was picked for its location due to very high compute intensity, multimodal type model requirements, given LA has a high propensity for video and all kinds of other things like that," Downs says, declining to share the exact location.

This first VC2 data center "has the ability to be a 10MW site," but is currently smaller. "And we will have racks very soon in a second building in LA, in Chicago, in Seattle, and in Columbus," Downs says.

In this first phase, the LA site will be home to some 40 SambaNova racks, SambaNova's VP of product, Anton McGonnell, discloses. Right now, they’re using the SN40 RDU, but are set to deploy the upcoming SN50 later this year.

"That's our next-gen system, which is very, very exciting for us, because we just believe we're going to dominate the fast decoding market on big models. We believe we're untouchable there,” McGonnell says.

For LLMs and other similar model types, inference is usually split into two parts. First is prefill, where prompt tokens are processed in parallel to build the attention key-value cache. Then comes decode, where the response is generated autoregressively, one token at a time.

For all of its talk of breaking the Nvidia stranglehold, even SambaNova has to defer to the GPU giant for part of the inference challenge. VC2 is set to co-deploy SambaRacks alongside Nvidia Blackwell GPUs, focusing on the prefill portion. Intel Xeon CPUs will meanwhile be used for orchestration and execution, with Intel a prominent investor in SambaNova.

"The SN50 is a good pre-filled machine as well," McGonnell argues. "But we've come to the conclusion that GPUs have this huge install base already, and are incredibly good prefill machines. With the compute crunch, the more we can alleviate that by accelerating GPUs, everybody ultimately wins."

Of course, Nvidia is itself trying to muscle in on the ultra-low latency space, with the not-quite-acquisition of Groq and the upcoming release of its Language Processor Unit (LPU).

"What Nvidia is trying to do with Groq - out of necessity more than the design - is separate decode into attention running on the GPUs and the FFN [Feed-Forward Network], where most of the weights reside, running on the LPUs for fast decoding. I think separating the FFN and attention just doesn’t make sense for a lot of technical reasons."

McGonnell was similarly skeptical about Cerebras, the big-chip company that recently went public at around $95 billion, but has steadily fallen since. “Cerebras being their own cloud, again, is out of necessity,” he says.

“Their systems are very, very hard for anybody else to deploy; it's very exotic, it needs its own proprietary liquid cooling system. Our systems are air-cooled and lightweight, can fit into the vast majority of existing brownfield data centers, and our software stack makes them very, very usable for existing cloud or data center operators. We don't want to build our own large-scale cloud; we want to sell racks and enable our partners to do it.”

It is hard to validate this critique against SambaNova’s hardware thanks to its current limited deployments. For all its positive talk, the company was forced to scrap its training effort and lay off 15 percent of its staff last year. While Intel has backed the business, a plan to sell the company to Intel also stalled.

However, the coming months will give SambaNova the chance to prove its lofty claims. VC2 is not the only one set to deploy SambaNova, with rival newcomer General Compute launching an inference service with multiple ASICs. However, it will be by far the largest deployment of the somewhat unproven technology.

Already, the effort comes pre-baked with potential customers, with Vista Equity offering it to its 90+ portfolio companies, some of which used it before its formal launch. "This is about being able to guarantee access to compute and low-cost compute for [our] portfolio, and we expect to use it heavily," Vista's head of strategy, Justin Hirsch, says.

Reflecting the increasingly interlinking nature of modern clouds, another customer is neocloud Together AI. “We continue to see exponential demand for inference tokens, now serving over 400T tokens a month of open models for agentic use cases," Together CEO Vipul Ved Prakash said earlier this year.

“We're not expecting to replace the need for anybody to use the existing clouds, but to augment them,” Downs says.

As token needs rise, so too may expectations of lower latency, he believes. “We think that there will be a continued premium for fast tokens and low latency. What will happen over time is that the expectation of what a commodity token is, from a speed perspective, will just change.

“Instead of it being 60 tokens a second right now, with fast mode 2.5x that, I could see what's fast mode today becoming a commodity, and the next step up is 400 tokens a second.”