The Field for Large Language Models

[The Bleeding Edge]( The Field for Large Language Models By Jeff Brown, Editor, The Bleeding Edge --------------------------------------------------------------- Standing amidst endless rows of head-high corn stalks, it just seems so out of place. Massive in size with industrial-scale cooling equipment outside and fenced-in with security cameras surrounding the facilities… these buildings are something that we’d expect to find in an urban industrial park. They just don’t seem to make any sense surrounded by rolling agricultural hills, residential homes, and farmsteads. And yet, these data centers are popping up everywhere… and seemingly always in the places we’d least expect to see them. But these locations are strategic. They all have two things in common: large amounts of cheap land and co-located with a power station that can provide cheap electricity. Thanks to fiber optic networks that can connect these data centers to the world at the speed of light, they can be constructed just about anywhere. They are part of the “cloud” – an amorphous industry word used to simply explain a data center located just about anywhere on Earth. If it’s “in the cloud,” that doesn’t mean it’s “up there in the sky somewhere,” it’s right here on Earth surrounded by green fields of future food. It’s about as real as it gets. The latest explosion in data center construction is grounded not by pie-in-the-sky projections, but in hardcore necessities. It’s founded in the physics and software needed to create intelligence. The story that has captured the media’s attention has been the torrid competition taking place in real-time to create the best-performing large language model (LLM). OpenAI kicked off the race in November 2022 with its release of ChatGPT. Since then, new LLM models have been popping up out of corn fields every couple of months since then. The latest round of LLMs is remarkable both in terms of what they are capable of, as well as what they cost. Billions in Investment… Each Just to train one of the leading models requires billions in investment in graphics processing units (GPUs), servers, racks, cooling systems, cabling, and of course the buildings to house it all. It also requires about 100 megawatts of electricity. That’s enough to power about 100,000 homes. Multiply that by at least 20 and we can get a sense of the scale of what’s happening. I agree, it is hard to get your head around it all. That’s why cheap land and cheap electricity is so important. Training a powerful artificial intelligence is expensive and resource-intensive… you need a lot of power and a lot of water to keep things running efficiently. There are now seven major players (ex-China), with five of them having well-established user interfaces – a technical term I use to describe their existing products that will act as immediate distribution for their AI technology… [(Click here to expand image)]( The performance of these models is tracked in real-time and measured against industry standard benchmarks. There has yet to be any “perfect” model. Some are better than others at certain tasks or tests. Performance Usually Comes at a Price One of the most tracked metrics is the overall performance quality measured against the cost to run that model. This is a highly relevant perspective. If a model is fantastic but too expensive to work with for most parties, then it simply won’t scale. Source: Artificial Analysis [(Click here to expand image)]( GPT-4o, which was just released May 13, was in a clear lead with Anthropic’s Claude 3.5 pretty close in terms of overall performance. That was until a few days ago on July 23 when Meta released Llama 3.1 405b which is on par for GPT-4o. GPT-4o stands alone as being the most expensive LLM to operate. That’s what makes Meta’s Llama 3.1 405b so impressive. Its performance is on par with OpenAI and is almost half as expensive to run. Noticeably absent from the analysis is xAI. Where Is xAI? Musk and his team at xAI released Grok 1.5 on April 12, but the industry acts like it doesn’t exist. The same treatment has happened in the electric car industry, where Tesla is largely ignored by industry publications and the current U.S. administration, which absurdly heralds General Motors as the leader in electric vehicles. The “problem”? xAI is working hard to develop an unbiased LLM that doesn’t attempt to redefine history, biology, math, or science. But xAI isn’t going anywhere. It’s a player after having raised $6 billion a few months ago. And just two weeks ago, it started training its next-generation model with enough horsepower to get it done. It has the GPUs… If having 100,000 liquid-cooled H100s weren’t enough, Musk and his teams always excel at technological architectures. That’s a fancy way of saying that Musk knows how to get a lot more out of less. It doesn’t matter if it is the system design for reusable rockets, electric vehicles, advanced batteries, autonomous driving software, brain-computer interface, tunnel-boring machinery, intelligent robotics, or training an artificial general intelligence (AGI). Musk figures out how to do it fast and efficiently. And we’ll see the impact of this on the cost to operate the new xAI LLM (Grok) once it has been trained. But these massive LLMs trained on hundreds of billions if not trillions of parameters are only part of the story, the story that gets all the attention… Running in parallel, emerging between stalks of corn, is a new breed of smaller language models designed for efficiency and cost, which is what we’ll be exploring tomorrow. --------------------------------------------------------------- Like what you’re reading? Send your thoughts to feedback@brownstoneresearch.com. [Brownstone Research]( Brownstone Research 55 NE 5th Avenue, Delray Beach, FL 33483 [www.brownstoneresearch.com]( To ensure our emails continue reaching your inbox, please [add our email address]( to your address book. This editorial email containing advertisements was sent to {EMAIL} because you subscribed to this service. To stop receiving these emails, click [here](. Brownstone Research welcomes your feedback and questions. But please note: The law prohibits us from giving personalized advice. To contact Customer Service, call toll free Domestic/International: 1-888-512-0726, Mon–Fri, 9am–7pm ET, or email us [here](mailto:memberservices@brownstoneresearch.com). © 2024 Brownstone Research. All rights reserved. Any reproduction, copying, or redistribution of our content, in whole or in part, is prohibited without written permission from Brownstone Research. [Privacy Policy]( | [Terms of Use](