When it comes to training large language models, the usual refrain is that the problem is a GPU shortage. Nvidia's dominant chips, of course, are the ones all the various AI outfits beat each other up to acquire.
For sure, getting hold of 100,000 H100s isn't going to be easy. Or cheap. But here's the thing. Each H100 eats up a peak of 700W of power. So, that's a peak of 70 megawatts for 100,000 of the things. OK, you're probably not going to have all 100,000 running at 100% load at the same time. But then there's more to an AI setup than just the GPUs. There's all kinds of supporting hardware and infrastructure involved.
So, yeah, 100 megawatts for just one LLM is a bit of a problem. And so in an interview with Norway wealth fund CEO), Musk stressed that while the availability of GPUs was and will continue to be a major constraint for the development of AI models, access to sufficient electricity will increasingly become a limiting factor.
But then he also predicted back in 2017 that self-driving cars reliable enough you could"go to sleep" in them were two years away. Still waiting on that one. And he predicted on March 19th 2020 that the US would have"close to zero new cases" of Covid19 by the end of April. Whoops!Sign up to get the best content of the week, and great gaming deals, as picked by the editors.