Replies

Transcript
0:00 · Artificial Intelligence consumes a lot of energy, both during training and during operation. We've
0:06 · heard a lot about this. Indeed, Sam Altman the CEO of OpenAI recently said that we'll need small
0:13 · modular nuclear reactors just to power all those AIs. Well, hold that thought. Today I want to
0:20 · look at how much energy these AIs really need and explain why I think this isn't the main problem.
0:31 · When we talk about how much energy AIs need, we have to distinguish between
0:36 · the training of the model and its regular use, in which user queries are worked off.
0:42 · The training is stunningly energy intensive. It's hard to get concrete numbers, but in 2022,
0:48 · a group of scientists estimated that training GPT-3 has eaten up at least 1300 megawatt
0:55 · hours. That's enough to power about 130 US homes for an entire year. And since then,
1:01 · large language models have only gotten larger.
1:04 · All that energy is expensive. Again, we don't have exact numbers, but some insiders have
1:09 · estimated that the training of GPT-4 has eaten up about 100 million dollars if not more. One
1:16 · way to get a sense of the cost is to have a look at the recent lawsuit of Musk against OpenAI.
1:22 · Yes, Elon Musk is suing OpenAI. It's a rather unfortunate fallout between the
1:27 · two parties which once worked together. In a recent blogpost on the matter, open AI writes
1:33 · "In early 2017, we came to the realization that building AGI will require vast quantities of
1:39 · compute. We began calculating how much compute an AGI might plausibly require. We all understood we
1:46 · were going to need a lot more capital to succeed at our mission—billions of dollars per year, which
1:52 · was far more than any of us, especially Elon, thought we'd be able to raise as the non-profit."
1:59 · Again we don't get any concrete numbers about whatever they estimated,
2:03 · but it gives you a sense of the money we're talking about: Billions per year.
2:08 · The story of the lawsuit is that OpenAI was originally set up as a non-profit. But that just
2:14 · didn't bring in the necessary money, so it was later restructured to a for-profit. Musk wanted
2:20 · OpenAI to become part of Tesla, OpenAI said no. Musk got out in 2018, Microsoft got in in 2019,
2:29 · OpenAI became a huge success in 2022. Now Musk wants part of the pie and is suing OpenAI over
2:36 · not fulfilling the original contract that said something about being a non-profit.
2:42 · You might find this a tangential drama, but it highlights just how huge sums of
2:47 · money we're talking about here. That's for the training, now let's talk about the operations.
2:53 · In December, a group from Hugging Face and Carnegie Mellon University published
2:58 · a pre-print with estimates for how much energy various AI models use for certain operations.
3:04 · This paper has not yet been peer-reviewed The authors ran tests on 88 different
3:10 · models for a number of different tasks, including prompts and image generation.
3:16 · They then estimated the energy use and the carbon dioxide emissions caused by that.
3:21 · A typical amount they found for text-use tasks is of the order of
3:25 · a few milliwatt hour per task, or a few Watt hours for a set of
3:30 · a thousand. For image generation however the amount is about 1000 times as high:
3:37 · Now we're talking about a few Watthours per item. According to the paper, that means if you use AI
3:42 · to generate one image that takes up almost as much energy as charging your smartphone. Oops!
3:49 · What does this mean for our future energy needs? According to the international energy agency,
3:55 · data centres now account for roughly one to percent of global electricity
4:00 · use. That's something in the ballpark of 400 Tera Watt hours energy per year,
4:06 · which is about as much as the energy consumption of the entire united kingdom.
4:10 · The increased use of Artificial Intelligence together with the continued crypto mining is
4:17 · likely to make data centres even more energy consuming. By 2026,
4:22 · the agency says, it could be more than twice as much as presently.
4:26 · Then again, there are many efforts underway to make AI more energy-efficient by using dedicated
4:33 · hardware that is particularly suited to the task. There is also a negative
4:38 · feedback that comes from using all these AI computations to make computations more
4:44 · energy efficient. A neat example for this comes from a few years back when
4:48 · Deepmind trained a system to help cool Google's data centres more efficiently.
4:54 · Then again, when everything is said and done,
4:56 · I expect the energy use to go up just because that's the way things normally
5:01 · go. So basically I agree with Altman, bring up the modular nuclear reactors.
5:09 · But there's a much more obvious problem hidden behind this energy intensity,
5:14 · which is cost. Building big AIs is so expensive and requires so much maintenance that eventually
5:22 · there will only be a few big ones globally owned by big companies or wealthy governments who don't
5:28 · want to rely on those companies. And most people will have a subscription to a private AI service.
5:35 · But how much we will our of them, well that will depend on how much we can pay. You can already
5:40 · see this trend happening right now that the more you want to do with an AI the higher the cost.
5:47 · Now imagine that we have a few AIs that have a fair shot at finding a theory of everything,
5:53 · or a cure for cancer, or figuring out exactly what you need to put into a tweet to get a reply from
5:58 · Elon Musk. But that'll take up an enormous amount of computing time. So, it'll be very expensive.
6:06 · The result will be that the rich will get richer and the poor will get poorer because
6:11 · they can't keep up. This is likely to happen both on an individual basis as well as on a national
6:17 · basis. And yeah, we could do something to prevent that from happening but I don't think we will.
6:24 · Hello
6:27 · Hi Elon, I told you to not call me on this number
6:33 · No, I'm not worried about AI at all, it's made the bots so much more interesting.
6:41 · Yes, and some of them sound just like Elon Musk!
6:48 · [ad text redacted]
7:56 · ...Thanks for watching, see you tomorrow.

Transcript
0:00	·	Artificial Intelligence consumes a lot of energy, both during training and during operation. We've
0:06	·	heard a lot about this. Indeed, Sam Altman the CEO of OpenAI recently said that we'll need small
0:13	·	modular nuclear reactors just to power all those AIs. Well, hold that thought. Today I want to
0:20	·	look at how much energy these AIs really need and explain why I think this isn't the main problem.
0:31	·	When we talk about how much energy AIs need, we have to distinguish between
0:36	·	the training of the model and its regular use, in which user queries are worked off.
0:42	·	The training is stunningly energy intensive. It's hard to get concrete numbers, but in 2022,
0:48	·	a group of scientists estimated that training GPT-3 has eaten up at least 1300 megawatt
0:55	·	hours. That's enough to power about 130 US homes for an entire year. And since then,
1:01	·	large language models have only gotten larger.
1:04	·	All that energy is expensive. Again, we don't have exact numbers, but some insiders have
1:09	·	estimated that the training of GPT-4 has eaten up about 100 million dollars if not more. One
1:16	·	way to get a sense of the cost is to have a look at the recent lawsuit of Musk against OpenAI.
1:22	·	Yes, Elon Musk is suing OpenAI. It's a rather unfortunate fallout between the
1:27	·	two parties which once worked together. In a recent blogpost on the matter, open AI writes
1:33	·	"In early 2017, we came to the realization that building AGI will require vast quantities of
1:39	·	compute. We began calculating how much compute an AGI might plausibly require. We all understood we
1:46	·	were going to need a lot more capital to succeed at our mission—billions of dollars per year, which
1:52	·	was far more than any of us, especially Elon, thought we'd be able to raise as the non-profit."
1:59	·	Again we don't get any concrete numbers about whatever they estimated,
2:03	·	but it gives you a sense of the money we're talking about: Billions per year.
2:08	·	The story of the lawsuit is that OpenAI was originally set up as a non-profit. But that just
2:14	·	didn't bring in the necessary money, so it was later restructured to a for-profit. Musk wanted
2:20	·	OpenAI to become part of Tesla, OpenAI said no. Musk got out in 2018, Microsoft got in in 2019,
2:29	·	OpenAI became a huge success in 2022. Now Musk wants part of the pie and is suing OpenAI over
2:36	·	not fulfilling the original contract that said something about being a non-profit.
2:42	·	You might find this a tangential drama, but it highlights just how huge sums of
2:47	·	money we're talking about here. That's for the training, now let's talk about the operations.
2:53	·	In December, a group from Hugging Face and Carnegie Mellon University published
2:58	·	a pre-print with estimates for how much energy various AI models use for certain operations.
3:04	·	This paper has not yet been peer-reviewed The authors ran tests on 88 different
3:10	·	models for a number of different tasks, including prompts and image generation.
3:16	·	They then estimated the energy use and the carbon dioxide emissions caused by that.
3:21	·	A typical amount they found for text-use tasks is of the order of
3:25	·	a few milliwatt hour per task, or a few Watt hours for a set of
3:30	·	a thousand. For image generation however the amount is about 1000 times as high:
3:37	·	Now we're talking about a few Watthours per item. According to the paper, that means if you use AI
3:42	·	to generate one image that takes up almost as much energy as charging your smartphone. Oops!
3:49	·	What does this mean for our future energy needs? According to the international energy agency,
3:55	·	data centres now account for roughly one to percent of global electricity
4:00	·	use. That's something in the ballpark of 400 Tera Watt hours energy per year,
4:06	·	which is about as much as the energy consumption of the entire united kingdom.
4:10	·	The increased use of Artificial Intelligence together with the continued crypto mining is
4:17	·	likely to make data centres even more energy consuming. By 2026,
4:22	·	the agency says, it could be more than twice as much as presently.
4:26	·	Then again, there are many efforts underway to make AI more energy-efficient by using dedicated
4:33	·	hardware that is particularly suited to the task. There is also a negative
4:38	·	feedback that comes from using all these AI computations to make computations more
4:44	·	energy efficient. A neat example for this comes from a few years back when
4:48	·	Deepmind trained a system to help cool Google's data centres more efficiently.
4:54	·	Then again, when everything is said and done,
4:56	·	I expect the energy use to go up just because that's the way things normally
5:01	·	go. So basically I agree with Altman, bring up the modular nuclear reactors.
5:09	·	But there's a much more obvious problem hidden behind this energy intensity,
5:14	·	which is cost. Building big AIs is so expensive and requires so much maintenance that eventually
5:22	·	there will only be a few big ones globally owned by big companies or wealthy governments who don't
5:28	·	want to rely on those companies. And most people will have a subscription to a private AI service.
5:35	·	But how much we will our of them, well that will depend on how much we can pay. You can already
5:40	·	see this trend happening right now that the more you want to do with an AI the higher the cost.
5:47	·	Now imagine that we have a few AIs that have a fair shot at finding a theory of everything,
5:53	·	or a cure for cancer, or figuring out exactly what you need to put into a tweet to get a reply from
5:58	·	Elon Musk. But that'll take up an enormous amount of computing time. So, it'll be very expensive.
6:06	·	The result will be that the rich will get richer and the poor will get poorer because
6:11	·	they can't keep up. This is likely to happen both on an individual basis as well as on a national
6:17	·	basis. And yeah, we could do something to prevent that from happening but I don't think we will.
6:24	·	Hello
6:27	·	Hi Elon, I told you to not call me on this number
6:33	·	No, I'm not worried about AI at all, it's made the bots so much more interesting.
6:41	·	Yes, and some of them sound just like Elon Musk!
6:48	·	[ad text redacted]
7:56	·	...Thanks for watching, see you tomorrow.