How much energy AI really needs. And why that's not its main problem. [8:05]

Transcript
0:00 · Artificial Intelligence consumes a lot of energy, both during training and during operation. We've
0:06 · heard a lot about this. Indeed, Sam Altman the CEO of OpenAI recently said that we'll need small
0:13 · modular nuclear reactors just to power all those AIs. Well, hold that thought. Today I want to
0:20 · look at how much energy these AIs really need and explain why I think this isn't the main problem.
0:31 · When we talk about how much energy AIs need, we have to distinguish between
0:36 · the training of the model and its regular use, in which user queries are worked off.
0:42 · The training is stunningly energy intensive. It's hard to get concrete numbers, but in 2022,
0:48 · a group of scientists estimated that training GPT-3 has eaten up at least 1300 megawatt
0:55 · hours. That's enough to power about 130 US homes for an entire year. And since then,
1:01 · large language models have only gotten larger.
1:04 · All that energy is expensive. Again, we don't have exact numbers, but some insiders have
1:09 · estimated that the training of GPT-4 has eaten up about 100 million dollars if not more. One
1:16 · way to get a sense of the cost is to have a look at the recent lawsuit of Musk against OpenAI.
1:22 · Yes, Elon Musk is suing OpenAI. It's a rather unfortunate fallout between the
1:27 · two parties which once worked together. In a recent blogpost on the matter, open AI writes
1:33 · "In early 2017, we came to the realization that building AGI will require vast quantities of
1:39 · compute. We began calculating how much compute an AGI might plausibly require. We all understood we
1:46 · were going to need a lot more capital to succeed at our mission—billions of dollars per year, which
1:52 · was far more than any of us, especially Elon, thought we'd be able to raise as the non-profit."
1:59 · Again we don't get any concrete numbers about whatever they estimated,
2:03 · but it gives you a sense of the money we're talking about: Billions per year.
2:08 · The story of the lawsuit is that OpenAI was originally set up as a non-profit. But that just
2:14 · didn't bring in the necessary money, so it was later restructured to a for-profit. Musk wanted
2:20 · OpenAI to become part of Tesla, OpenAI said no. Musk got out in 2018, Microsoft got in in 2019,
2:29 · OpenAI became a huge success in 2022. Now Musk wants part of the pie and is suing OpenAI over
2:36 · not fulfilling the original contract that said something about being a non-profit.
2:42 · You might find this a tangential drama, but it highlights just how huge sums of
2:47 · money we're talking about here. That's for the training, now let's talk about the operations.
2:53 · In December, a group from Hugging Face and Carnegie Mellon University published
2:58 · a pre-print with estimates for how much energy various AI models use for certain operations.
3:04 · This paper has not yet been peer-reviewed The authors ran tests on 88 different
3:10 · models for a number of different tasks, including prompts and image generation.
3:16 · They then estimated the energy use and the carbon dioxide emissions caused by that.
3:21 · A typical amount they found for text-use tasks is of the order of
3:25 · a few milliwatt hour per task, or a few Watt hours for a set of
3:30 · a thousand. For image generation however the amount is about 1000 times as high:
3:37 · Now we're talking about a few Watthours per item. According to the paper, that means if you use AI
3:42 · to generate one image that takes up almost as much energy as charging your smartphone. Oops!
3:49 · What does this mean for our future energy needs? According to the international energy agency,
3:55 · data centres now account for roughly one to percent of global electricity
4:00 · use. That's something in the ballpark of 400 Tera Watt hours energy per year,
4:06 · which is about as much as the energy consumption of the entire united kingdom.
4:10 · The increased use of Artificial Intelligence together with the continued crypto mining is
4:17 · likely to make data centres even more energy consuming. By 2026,
4:22 · the agency says, it could be more than twice as much as presently.
4:26 · Then again, there are many efforts underway to make AI more energy-efficient by using dedicated
4:33 · hardware that is particularly suited to the task. There is also a negative
4:38 · feedback that comes from using all these AI computations to make computations more
4:44 · energy efficient. A neat example for this comes from a few years back when
4:48 · Deepmind trained a system to help cool Google's data centres more efficiently.
4:54 · Then again, when everything is said and done,
4:56 · I expect the energy use to go up just because that's the way things normally
5:01 · go. So basically I agree with Altman, bring up the modular nuclear reactors.
5:09 · But there's a much more obvious problem hidden behind this energy intensity,
5:14 · which is cost. Building big AIs is so expensive and requires so much maintenance that eventually
5:22 · there will only be a few big ones globally owned by big companies or wealthy governments who don't
5:28 · want to rely on those companies. And most people will have a subscription to a private AI service.
5:35 · But how much we will our of them, well that will depend on how much we can pay. You can already
5:40 · see this trend happening right now that the more you want to do with an AI the higher the cost.
5:47 · Now imagine that we have a few AIs that have a fair shot at finding a theory of everything,
5:53 · or a cure for cancer, or figuring out exactly what you need to put into a tweet to get a reply from
5:58 · Elon Musk. But that'll take up an enormous amount of computing time. So, it'll be very expensive.
6:06 · The result will be that the rich will get richer and the poor will get poorer because
6:11 · they can't keep up. This is likely to happen both on an individual basis as well as on a national
6:17 · basis. And yeah, we could do something to prevent that from happening but I don't think we will.
6:24 · Hello
6:27 · Hi Elon, I told you to not call me on this number
6:33 · No, I'm not worried about AI at all, it's made the bots so much more interesting.
6:41 · Yes, and some of them sound just like Elon Musk!
6:48 · [ad text redacted]
7:56 · ...Thanks for watching, see you tomorrow.

Oh, no, inequality! Heavens to Betsy.

Altman is sus. I don’t even know they have the gall to call it OpenAI at this point.

I believe that nvidea’s H100 tensor core runs at a max of 700 watts each. That’s a lot of power and cooling requirements.

And we have a proven, cheap, and reliably source of energy which is off limits because too many people who are too stupid to understand basic math and science are allowed to vote.

Enough with the videos. jmo

This AI garbage is going to take all of our water someday. What stupid animals humans are. Who needs this AI sh#t??? No one, that’s who.

Ditto. I don’t bother with videos.

“I believe that nvidea’s H100 tensor core runs at a max of 700 watts each. That’s a lot of power and cooling requirements.”

About 800 to equal the power of my car.

“I believe that nvidea’s H100 tensor core runs at a max of 700 watts each. That’s a lot of power and cooling requirements.”

If you consider heat and mechanical losses my car consumes about about the same energy as 4000. When the pedal hits the metal.

“This video isn’t available anymore”

It could be she’s had to take it down to edit something.

Google/YouTube has been censoring her off and on, she’s not left enough for the cancel culture.

It seems to be up here.

There’s a cycle to the water, fortunately.

Video gone already.

Ok maybe I’m being paranoid. I just don’t get why people love that AI stuff.

Transcript
0:00	·	Artificial Intelligence consumes a lot of energy, both during training and during operation. We've
0:06	·	heard a lot about this. Indeed, Sam Altman the CEO of OpenAI recently said that we'll need small
0:13	·	modular nuclear reactors just to power all those AIs. Well, hold that thought. Today I want to
0:20	·	look at how much energy these AIs really need and explain why I think this isn't the main problem.
0:31	·	When we talk about how much energy AIs need, we have to distinguish between
0:36	·	the training of the model and its regular use, in which user queries are worked off.
0:42	·	The training is stunningly energy intensive. It's hard to get concrete numbers, but in 2022,
0:48	·	a group of scientists estimated that training GPT-3 has eaten up at least 1300 megawatt
0:55	·	hours. That's enough to power about 130 US homes for an entire year. And since then,
1:01	·	large language models have only gotten larger.
1:04	·	All that energy is expensive. Again, we don't have exact numbers, but some insiders have
1:09	·	estimated that the training of GPT-4 has eaten up about 100 million dollars if not more. One
1:16	·	way to get a sense of the cost is to have a look at the recent lawsuit of Musk against OpenAI.
1:22	·	Yes, Elon Musk is suing OpenAI. It's a rather unfortunate fallout between the
1:27	·	two parties which once worked together. In a recent blogpost on the matter, open AI writes
1:33	·	"In early 2017, we came to the realization that building AGI will require vast quantities of
1:39	·	compute. We began calculating how much compute an AGI might plausibly require. We all understood we
1:46	·	were going to need a lot more capital to succeed at our mission—billions of dollars per year, which
1:52	·	was far more than any of us, especially Elon, thought we'd be able to raise as the non-profit."
1:59	·	Again we don't get any concrete numbers about whatever they estimated,
2:03	·	but it gives you a sense of the money we're talking about: Billions per year.
2:08	·	The story of the lawsuit is that OpenAI was originally set up as a non-profit. But that just
2:14	·	didn't bring in the necessary money, so it was later restructured to a for-profit. Musk wanted
2:20	·	OpenAI to become part of Tesla, OpenAI said no. Musk got out in 2018, Microsoft got in in 2019,
2:29	·	OpenAI became a huge success in 2022. Now Musk wants part of the pie and is suing OpenAI over
2:36	·	not fulfilling the original contract that said something about being a non-profit.
2:42	·	You might find this a tangential drama, but it highlights just how huge sums of
2:47	·	money we're talking about here. That's for the training, now let's talk about the operations.
2:53	·	In December, a group from Hugging Face and Carnegie Mellon University published
2:58	·	a pre-print with estimates for how much energy various AI models use for certain operations.
3:04	·	This paper has not yet been peer-reviewed The authors ran tests on 88 different
3:10	·	models for a number of different tasks, including prompts and image generation.
3:16	·	They then estimated the energy use and the carbon dioxide emissions caused by that.
3:21	·	A typical amount they found for text-use tasks is of the order of
3:25	·	a few milliwatt hour per task, or a few Watt hours for a set of
3:30	·	a thousand. For image generation however the amount is about 1000 times as high:
3:37	·	Now we're talking about a few Watthours per item. According to the paper, that means if you use AI
3:42	·	to generate one image that takes up almost as much energy as charging your smartphone. Oops!
3:49	·	What does this mean for our future energy needs? According to the international energy agency,
3:55	·	data centres now account for roughly one to percent of global electricity
4:00	·	use. That's something in the ballpark of 400 Tera Watt hours energy per year,
4:06	·	which is about as much as the energy consumption of the entire united kingdom.
4:10	·	The increased use of Artificial Intelligence together with the continued crypto mining is
4:17	·	likely to make data centres even more energy consuming. By 2026,
4:22	·	the agency says, it could be more than twice as much as presently.
4:26	·	Then again, there are many efforts underway to make AI more energy-efficient by using dedicated
4:33	·	hardware that is particularly suited to the task. There is also a negative
4:38	·	feedback that comes from using all these AI computations to make computations more
4:44	·	energy efficient. A neat example for this comes from a few years back when
4:48	·	Deepmind trained a system to help cool Google's data centres more efficiently.
4:54	·	Then again, when everything is said and done,
4:56	·	I expect the energy use to go up just because that's the way things normally
5:01	·	go. So basically I agree with Altman, bring up the modular nuclear reactors.
5:09	·	But there's a much more obvious problem hidden behind this energy intensity,
5:14	·	which is cost. Building big AIs is so expensive and requires so much maintenance that eventually
5:22	·	there will only be a few big ones globally owned by big companies or wealthy governments who don't
5:28	·	want to rely on those companies. And most people will have a subscription to a private AI service.
5:35	·	But how much we will our of them, well that will depend on how much we can pay. You can already
5:40	·	see this trend happening right now that the more you want to do with an AI the higher the cost.
5:47	·	Now imagine that we have a few AIs that have a fair shot at finding a theory of everything,
5:53	·	or a cure for cancer, or figuring out exactly what you need to put into a tweet to get a reply from
5:58	·	Elon Musk. But that'll take up an enormous amount of computing time. So, it'll be very expensive.
6:06	·	The result will be that the rich will get richer and the poor will get poorer because
6:11	·	they can't keep up. This is likely to happen both on an individual basis as well as on a national
6:17	·	basis. And yeah, we could do something to prevent that from happening but I don't think we will.
6:24	·	Hello
6:27	·	Hi Elon, I told you to not call me on this number
6:33	·	No, I'm not worried about AI at all, it's made the bots so much more interesting.
6:41	·	Yes, and some of them sound just like Elon Musk!
6:48	·	[ad text redacted]
7:56	·	...Thanks for watching, see you tomorrow.