Training a model takes more power than what? Generating a single poem? Using it to generate an entire 4th grade class’s essays? To answer all questions in Hawaii for 6th months? What is the scale? The break even point for training is far far less than total usage.
Have you ever used one locally? Depending on your hardware it’s anywhere between glacially to a morgue’s AC slow. To the average person on the average computer it is nearly unusable, relative to the instant gratification of the web interface.
That gives you a sense of the resources required to do the task at all, but it doesn’t scale linearly. 2 computers aren’t twice as fast as one. It’s logarithmic. With diminishly returns. In the end, this means one 100 word response uses the equivalent of 3 bottles of water.
How many queries are made per hour? How does that scale over time with increased usage of the same model? More than training a model. A lot more.
Yeah you make a really good point there! I was perhaps thinking too simplistically and scaling from my personal experience with playing around on my home machine.
Although realistically, it seems the situation is pretty bad because freaky-giant-mega-computers are both training models AND answering countless silly queries per second.
So at scale it sucks all around.
Minus the terrible fad-device-cycle manufacturing aspect, if they’re really sticking to their guns on pushing this LLM madness, do you think this wave of onboard “Ai chips” will make any impact on lessening natural resource usage at scale?
(Also offtopic but I wonder how much a sweet juicy exploit target these “ai modules” will turn out to be.)
It’s really opaque. We won’t know the environmental impact right away. Part of the larger problem is, while folks like you and I make a sizable impact, it’s nothing compared to enterprise usage at scale. Every website, app, and operating system with an AI button makes it even easier for users to interface with AI leading to more queries. Not only that, those queries and responses are collected and used to further make queries.
Should the usage of AI stay stable, improved hardware would decrease carbon output. We should be cautious coming to that conclusion. What is more likely is that increased efficiency will lead to increased usage. Perhaps at an accelerated rate with the anticipation of even more technological breakthroughs down the line.
All that said, I’m really not a doomer. It’s important we all consider the cost of our choices. The way I see it, we are all going to die eventually. I’m old enough it will probably be from something else.
Training a model takes more power than what? Generating a single poem? Using it to generate an entire 4th grade class’s essays? To answer all questions in Hawaii for 6th months? What is the scale? The break even point for training is far far less than total usage.
Have you ever used one locally? Depending on your hardware it’s anywhere between glacially to a morgue’s AC slow. To the average person on the average computer it is nearly unusable, relative to the instant gratification of the web interface.
That gives you a sense of the resources required to do the task at all, but it doesn’t scale linearly. 2 computers aren’t twice as fast as one. It’s logarithmic. With diminishly returns. In the end, this means one 100 word response uses the equivalent of 3 bottles of water.
How many queries are made per hour? How does that scale over time with increased usage of the same model? More than training a model. A lot more.
Yeah you make a really good point there! I was perhaps thinking too simplistically and scaling from my personal experience with playing around on my home machine.
Although realistically, it seems the situation is pretty bad because freaky-giant-mega-computers are both training models AND answering countless silly queries per second. So at scale it sucks all around.
Minus the terrible fad-device-cycle manufacturing aspect, if they’re really sticking to their guns on pushing this LLM madness, do you think this wave of onboard “Ai chips” will make any impact on lessening natural resource usage at scale?
(Also offtopic but I wonder how much a sweet juicy exploit target these “ai modules” will turn out to be.)
It’s really opaque. We won’t know the environmental impact right away. Part of the larger problem is, while folks like you and I make a sizable impact, it’s nothing compared to enterprise usage at scale. Every website, app, and operating system with an AI button makes it even easier for users to interface with AI leading to more queries. Not only that, those queries and responses are collected and used to further make queries.
Should the usage of AI stay stable, improved hardware would decrease carbon output. We should be cautious coming to that conclusion. What is more likely is that increased efficiency will lead to increased usage. Perhaps at an accelerated rate with the anticipation of even more technological breakthroughs down the line.
All that said, I’m really not a doomer. It’s important we all consider the cost of our choices. The way I see it, we are all going to die eventually. I’m old enough it will probably be from something else.