Is ChatGPT or Google AI better?

Hey y’all, it’s Austin Carr in Boston. The battle between Google and OpenAI for AI supremacy is a little bewildering. But first...Three thin [View in browser]( [Bloomberg]( [by Austin Carr]( Hey y’all, it’s Austin Carr in Boston. The battle between Google and OpenAI for AI supremacy is a little bewildering. But first... Three things you need to know today: • Microsoft’s OpenAI deal [drew US and UK inquiries]( • An Apple product design [chief will leave]( • Baldur’s Gate 3 was named “[game of the year](” What’s your sign? Google has a new artificial intelligence model called [Gemini](, and to prove to the world it’s better than the one used by ChatGPT, the company employed the most boring means possible: a report card. In a company blog post co-written by Chief Executive Officer Sundar Pichai, Google argues Gemini outperforms OpenAI Inc.’s GPT-4 in 30 of 32 academic benchmarks, including tests for math, reading, coding and reasoning. Gemini is certainly a [huge improvement]( over where Google was early this year, but the boasts seemed a tad desperate, especially since Google was only ahead of OpenAI’s model by a few percentage points or less on many of the tests. Is anyone really going to switch to Bard from ChatGPT because Google scored 0.3% better on an assessment of algebra skills? With so many AI products trying to distinguish themselves, the points of comparison are abstract to the point of being almost meaningless. Tech giants like Baidu Inc. and Meta Platforms Inc. point to how their large language models boast billions, if not trillions, of parameters. Startups tout the millions of pieces of content their systems have ingested. For supercomputer operators, it’s the tens of thousands of AI chips they’ve strung together. Even experts in the field are wary of these kinds of evaluations. Rowan Zellers, an OpenAI researcher who helped develop a test for commonsense tasks called HellaSwag, [posted]( on the social network X after Gemini’s unveiling saying he doesn’t have “a good sense on how much to trust the dozen or so text benchmarks that all the LLM papers report on these days.” What matters more than these test grades is how Gemini [performs in the wild](, and users were quick to point out that the system is [still prone to errors]( and seemingly [no better than OpenAI’s]( offering. Google itself had difficulty translating exactly what Gemini’s scores meant for the public, calling the model its “most flexible” and “most capable” and “largest” ever. Never mind that it comes in three distinct versions — Ultra, Pro and Nano — that even Google’s Bard had trouble identifying. When I asked the chatbot about its new Gemini model, it directed me to various products named Gemini Pro, including a high-gloss floor finish, portable PA system and an embossing machine. The excessive benchmarking looks a bit like the spec wars of the PC era. Decades ago, computer manufacturers promoted their hardware with wonky metrics related to processor speeds and memory size. Apple Inc. is credited with shifting the focus to novel features that demonstrated how a product would enhance your daily life. When Steve Jobs unveiled the first iPod, its gigabytes of storage mattered far less than that the promise of “1,000 songs in your pocket.” The problem for AI companies is that, with their software claiming to be able to do pretty much anything, it’s really hard to distill exactly what makes a particular product special. Elon Musk, for his part, pitches xAI’s [Grok]( as an anti-woke alternative, and [X CEO Linda Yaccarino calls it]( “the ultimate ride or die” (whatever that means). Google’s clearest differentiator was showed off in a science fair-like demo of Bard’s [“multimodal” features](, meaning how you can interact with the machine outside text inputs. In one example, Bard saw an image of a forked road with lanes leading to either a doodle of a duck or a bear. Asked which direction another duck at the crossroads should go, Bard correctly guessed the safest path was toward its kin. But Google later acknowledge the demo was [sort of fudged]( and based on an unreleased version of Gemini. When I ran the same test on Bard, it oddly recommended the duck should take the highway 100 miles south toward Rhode Island on a bicycle. ChatGPT, on the other hand, nailed the prompt perfectly. Google will need to work harder to earn a passing grade on this one. —[Austin Carr](mailto:acarr54@bloomberg.net) The big story Microsoft’s answer to regulatory inquiries about its relationship with OpenAI is that it [doesn’t own a stake in the company](. Instead, it will receive almost half of OpenAI’s financial returns, an unorthodox structure designed to accommodate the startup’s nonprofit origin. Regulators may not care to see a distinction. One to watch [Watch the Bloomberg Technology interview]( with Jennifer Doudna, a co-inventor of Crispr and 2020 Nobel Prize winner. Get fully charged Amazon is seeking the dismissal of [US regulators’ antitrust lawsuit](. Microsoft will convert temp workers on its video game staff [to unionized employees](. Google said the EU’s proposal to [break up its profitable ad tech arm]( was “flawed.” More from Bloomberg Get Bloomberg Tech weeklies in your inbox: - [Cyber Bulletin]( for coverage of the shadow world of hackers and cyber-espionage - [Game On]( for reporting on the video game business - [Power On]( for Apple scoops, consumer tech news and more - [Screentime]( for a front-row seat to the collision of Hollywood and Silicon Valley - [Soundbite]( for reporting on podcasting, the music industry and audio trends - [Q&AI]( for answers to all your questions about AI Follow Us Like getting this newsletter? [Subscribe to Bloomberg.com]( for unlimited access to trusted, data-driven journalism and subscriber-only insights. Want to sponsor this newsletter? [Get in touch here](. You received this message because you are subscribed to Bloomberg's Tech Daily newsletter. If a friend forwarded you this message, [sign up here]( to get it in your inbox. [Unsubscribe]( [Bloomberg.com]( [Contact Us]( Bloomberg L.P. 731 Lexington Avenue, New York, NY 10022 [Ads Powered By Liveintent]( [Ad Choices](