The AI industry’s soaring electricity demand has already become a growing concern for governments, utilities, and technology companies. But a new study suggests the next generation of artificial intelligence could make that problem significantly worse.
Researchers from the Korea Advanced Institute of Science and Technology (KAIST) have published what they describe as the first comprehensive analysis of the energy cost of AI agents – AI systems capable of reasoning, planning, and completing tasks autonomously. Their findings show that these systems can consume up to 136.5 times as much energy per query as conventional generative AI models, raising fresh questions about whether the infrastructure supporting tomorrow’s AI is ready for what’s coming.
Smarter AI comes with a much steeper electricity bill
Unlike traditional chatbots that generate a single response to a prompt, AI agents repeatedly call large language models (LLMs), browse the web, execute code, use calculators, and interact with external software while solving complex tasks. While these capabilities make them significantly more useful for research, programming, and workplace automation, they also require far more computing resources.
Led by Professor Minsoo Rhu from KAIST’s School of Electrical Engineering, the research team treated AI agents as a new category of data center workload. It measured their computational requirements in real-world scenarios.
The results were striking. The researchers found that AI agents can increase response latency by up to 153.7 times compared to conventional chain-of-thought reasoning. More surprisingly, the expensive GPUs powering these workloads remained idle for up to 54.5 percent of execution time while waiting for external tools to finish their tasks. In other words, the hardware continues consuming power even when it isn’t actively performing AI computation.
Energy usage scales just as dramatically. Running an AI agent powered by a 70-billion-parameter language model, similar in size to today’s commercial AI systems, required an average of 348.41 watt-hours per query. That’s roughly 136.5 times higher than a conventional chatbot answering a straightforward question.
To understand the broader implications, the team modelled a future where AI agents handle 13.7 billion requests per day – roughly equivalent to Google’s daily search traffic. Under that scenario, AI infrastructure would require approximately 198.9 gigawatts of electricity, nearly half of the average power consumed across the entire United States and far beyond the capacity of today’s AI data centers.
The hidden cost of intelligence
The findings arrive as companies including OpenAI, Google, Microsoft, Anthropic, and others increasingly invest in agentic AI, positioning it as the next major leap beyond conversational chatbots. But the study argues that improving AI models alone is no longer enough. Future progress will depend equally on more efficient semiconductors, better GPU utilization, smarter data-center design, and expanded power infrastructure.
Professor Rhu says the research demonstrates that AI competitiveness is shifting from building “smarter AI” to building more efficient AI. The team believes future AI development will require a co-design approach, optimizing models, AI chips, servers, and energy systems together to keep operating costs manageable and ensure AI remains sustainable at scale.
The paper, titled “The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective,” was presented at the IEEE International Symposium on High-Performance Computer Architecture (HPCA) earlier this year. The researchers have also open-sourced their AI agent benchmarks, hoping to encourage further work on reducing one of AI’s fastest-growing—and often overlooked—costs: electricity.
