Energy Efficient Large Language Models: Advancements and Challenges

Energy Efficient Large Language Models: Advancements and Challenges

Vishakha Agrawal vishakha.research.id@gmail.com

Abstract—The rapid growth of Large Language Models (LLMs) has revolutionized the field of natural language process- ing (NLP), achieving unparalleled levels of linguistic nuance and precision. The debut of ChatGPT, a cutting-edge LLM, has cat- alyzed widespread adoption across diverse applications. Built on the GPT-3.5 architecture, ChatGPT’s 175 billion parameters were honed through reinforcement learning from human feedback, yielding remarkable performance. Nevertheless, the escalating scale and intricacy of LLMs, epitomized by ChatGPT’s substan- tial parameter count, have sparked pressing concerns regarding their ecological footprint and energy expenditure. As demand for LLMs continues to surge, developing sustainable, energy-efficient solutions is crucial to mitigating their environmental impact. This paper undertakes a comprehensive examination of energy consumption monitoring, challenges, and areas for improvement necessary for the development of energy efficient Large Language Models.

Index Terms—BERT, Large Language Model, Energy-aware training, GPU, Footprint, GPT-3, Profiling, CO2 emissions

DOI: 10.55041/IJSREM11170