Nvidia has put its name behind a fledgling effort to put mini-data centers beside people’s homes in boxes that look like HVAC units. It’s a “power” play, considering that the main bottleneck to building out more data center capacity is not money or chips, but rather retrofitting the electrical grid to supply the power.
The idea, put forward by a California smart utility box company called Span, is to put the GPUs where the power has already been allocated—at the home. Span says the average household uses only about 42% of the electricity allotted to it, and rarely reaches peak usage. Span’s smart utility boxes detect that, and steer the extra available power over to the GPUs, which live inside a “node” that sits beside the house and looks something like an HVAC unit. The boxes contain 16 Nvidia GPUs, 4 AMD CPUs, 4 terabytes of memory, and a cooling system. When a large number of homes have these, the servers could be connected together in a network and work together on distributed computing jobs (workloads), Span says.
In exchange for hosting a node, Span pays a big chunk of the homeowner’s electricity and broadband internet bills.
And there may even be advantages for putting the compute power closer to the end users that are using the chatbots or AI services, Span says.
It’s a cool idea on paper, but it’s almost completely unproven in real-world use. Span has been prototyping the units but has yet to install any of them beside real homes. I asked Span VP Chris Lander if his company has done technical studies showing that its brand of distributed computing will be fast and robust enough to handle real AI workloads. “We’ve done a bunch of technical studies internally [and] a bunch of modeling for different kinds of workloads, both from the business point of view [and] the product point of view and from the technical architecture point of view,” he replies.
The company is working with a homebuilder, Atlanta-based Pulte Homes, to build the nodes at new homes, but Pulte told CNBC that it’s so far put a Span unit next to exactly one home. “I will say that we’ve been collaborating with Pulte amongst others to test the latest proof of concept design, the latest prototype that we have,” Lander says. Span says it’ll have “upwards of 100” nodes of an advanced version of its prototype in a pilot project “later this year,” but isn’t saying when or where the pilot will be built.
The main point of resistance to new data centers across the country is the risk that the facilities will result in higher electric bills for everyone in the area. Whether it’s a new central data center or a distributed data center, as Span proposes, that’s drawing more power from the grid, the risk of higher costs—perhaps because of transformers and other infrastructure running hotter and degrading more quickly—could arguably be the same.
Lander disagrees. “We believe it’s actually going to be the opposite—that it’s actually going to give relief to customers, not just for those direct customers that are hosting, that we’re paying for their energy, but by allaying some of that additional CapEx spend that utilities would have to pay to build out [more] data centers,” he says.
The existence of Span’s idea, which has gotten a good deal of attention in the press and on social media, is one of the first signs of the market finding ways to address the serious dearth of data center capacity needed to support the expected demand for artificial intelligence services. Data centers take time to build and often face political resistance, and the demand for AI compute is growing now.
On the political side, crowdsourcing already-provisioned power from households in a community may be easier than talking a city council into issuing a permit for a data center. As Reddit user unicynicist put it: “It’s like Uber, but for turning your house into someone else’s unpermitted data center.”
Still, it says something that Nvidia allowed Span to include its brand in the press release, but the GPU maker’s involvement beyond that has been mostly consultation. “Nvidia’s been a thought partner, and they’ve been helping us from a business point of view to connect us with the right folks.” But Nvidia is not an investor and has so far not donated any GPUs to Span’s initiative.
And the chips are not cheap. In fact, the Span box contains some very expensive hardware. The chips and other technology could be worth $500,000 or more, based on available pricing information for the specific components. The risk of theft then becomes an issue.
If the home-side server box concept proves out and produces a meaningful amount of compute resources, Nvidia could help Span find potential buyers. In fact, that effort is already underway. “We’ve had conversations with the breadth of potential compute offtakers, whether they’re hyperscalers, neoclouds, neoscalers, and AI service providers,” Lander says.