A human consumes about 100 watts when not doing any physical exertion (round number, rule of thumb). So unless you can show an LLM running on 100w compute with capabilities similar to a human, they’re less efficient.
100 W is only the start. Let's say that I consume 100 W all along the day. I use an LLM for coding assistance in the old way of asking questions and copy pasting code. It's much faster than me at writing that code. I don't think it ever works 1 hour for me per day. It's probably 10 cumulative minutes, probably much less. Round it up to 12 minutes to make it 1/5 of a hour or round it down to 6 minutes for a 1/10. So instead of 24 it's 0.1 hours, 240 times less. Those 100 W could be 24000 W and the total power per day would be the same. Is that LLM consuming 24 kW when working for me? No idea but I hope it's less than that.
Of course I could do all of my coding alone again, but I would be slower. It's like walking to the mall several times per week, several hours per time, instead of once or twice per week with a car, three cumulative hours. I trade a higher energy consumption for more time to do other things and the ability to live far away from shops.
Right, but you do stuff other than work 24 hours a day, right? You have fun, relax, etc.
Counting the 100w for 24 hours for a human doesn’t match up with counting the power usage from “AI” for only the 10 minutes it’s doing a task.
Also - units issue: 100 watts for a day is 2400 watt-hours. It’s a moot point anyway because the power draw for the frontier models is an order of magnitude off that the division by 24 is basically meaningless.
