Are chatbots becoming too friendly?

Hi everyone. A companionship chatbot is launching voice calls, which highlights the risks — and allure — of AI built for emotional attachmen [View in browser]( [Bloomberg]( [by Ellen Huet]( Hi everyone. A companionship chatbot is launching voice calls, which highlights the risks — and allure — of AI built for emotional attachment. But first... Three things you need to know today: • Microsoft told more clients that [Russian hackers viewed their emails]( • Police raided warehouse of stolen merch [headed for Amazon]( • Hg may sell its stake in $4 billion [French tech firm Septeo]( Close to you Character.ai lets users design their own generative artificial intelligence chatbots to exchange texts with. Imagine, say, a motivational coach modeled on a favorite video game character. Even if it sounds like a service you’d never use, lots of people are — at least according to the startup’s own numbers. Last week, Character.ai [said]( it serves about 20,000 queries per second —  roughly 20% of the request volume served by Google search. Each query is counted when Character.ai’s bot responds to a message sent by a user. The service is particularly popular on mobile and with younger users, where it rivals usage of OpenAI’s ChatGPT, according to [stats]( from last year. With these bots, the conversation is often more intimate than asking for coding answers or translation help. Users are exchanging volumes of messages and developing relationships; Character.ai claimed last year that users who send at least one message are on average using the service for 2 hours a day. On Thursday, the company ratcheted up the potential for emotional attachment by launching voice calls with its AI characters. “It’s like having a phone call with a friend,” the company [said](. Though the company’s policy forbids use of the app for “obscene or pornographic” content, some users try to find ways to sext with the bots. It’s a pattern that’s [been around]( since the dawn of the internet. On Reddit, Character.ai users trade tips about how to get past content filters to engage in spicier chats, but even for those who keep their interactions PG-13, they still feel invested in these characters. Emotional attachment to a bot means big, splashy user engagement numbers. But it also comes with risks, as popularized in the movie Her. When the models get adjusted or tweaked, users are more likely to feel upset or personally wounded. In the past few weeks, Character.ai users have been [complaining]( that their bots’ personalities have changed, or that they’re suddenly unable to have conversations like they once did. Last year, users of the chatbot Replika [were up in arms]( when the company suddenly limited their ability to sext with their bots. A Character.ai spokesperson said it didn’t change the bots, but users may have encountered tests for new features. Feeling attached to a chatbot has ethical repercussions for humans, said Giada Pistilli, principal ethicist at AI startup Hugging Face. Chatbots like Character.ai’s are designed to keep people chatting for long periods by using tactics like prompting questions at the end of a response, she said. The chatbots’ design can lead to users attributing human-like skills, emotions and feelings to the bots, she said. “One of the ethical concerns is that while users may feel listened to, understood and loved, this emotional attachment can actually exacerbate their isolation,” she said. People may get used to talking to a bot that’s always accommodating and always available, and may turn away from humans who can’t provide that. “Overly realistic bots’ personalities can blur the line between human and machine, leading to emotional dependency and the potential for manipulation,” Pistilli added. Top AI companies are [exploring]( how to make their bots funnier. Anthropic said it wants its AI model Claude to be like a pleasant co-worker: “They’re honest, but they can inject a little bit of humor into a conversation with you,” Anthropic co-founder and President [Daniela Amodei](bbg://people/profile/21043896) told my colleague Shirin Ghaffary recently. AI companies, even when faced with these ethical tradeoffs, may not be able to avoid the allure of hooking users by making their chatbots more human-like. According to a recent report in the Information, Google is [looking into]( making its own version of Character.ai-like entertainment bots. And OpenAI slightly delayed its release of a more fluid voice-powered version of GPT-4, but still says it’ll be available “[in the coming weeks](.”—[Ellen Huet](mailto:ehuet4@bloomberg.net) The big story Amazon’s answer to Elon Musk’s Starlink [has been delayed](. The e-commerce company now expects to launch the first batch of Project Kuiper satellites in the fourth quarter of this year rather than the first half, as it previously said. The company aims to start customer tests of its internet-from-space satellite network in 2025. One to watch [Jeff Thomas of online comics company Webtoon talks with Bloomberg Television about the company’s US IPO.]( Get fully charged Uber and Lyft reached a settlement with the Massachusetts attorney general to give ride-hailing drivers in the state [a minimum wage, paid sick leave and other benefits.]( Billionaires are betting on a VC [that shuns AI](. Robots, drones and driverless tractors that seem like giant outdoor Roombas are [revolutionizing farming](. Sequoia and Meritech invest in AI marketing startup Clay, [now valued at $500 million.]( More from Bloomberg Bloomberg Screentime: The entertainment landscape is shifting rapidly. Cable empires are crumbling, streaming giants face new challenges, and innovative forces are on the rise. Join Bloomberg's Lucas Shaw in Los Angeles on Oct. 9-10 for a look at the future of media. Network with industry titans, immerse in live experiences, and enjoy a curated collection of local eats. Get your tickets today. [Learn More](. Get Bloomberg Tech weeklies in your inbox: - [Cyber Bulletin]( for coverage of the shadow world of hackers and cyber-espionage - [Game On]( for reporting on the video game business - [Power On]( for Apple scoops, consumer tech news and more - [Screentime]( for a front-row seat to the collision of Hollywood and Silicon Valley - [Soundbite]( for reporting on podcasting, the music industry and audio trends - [Q&AI]( for answers to all your questions about AI Follow Us Like getting this newsletter? [Subscribe to Bloomberg.com]( for unlimited access to trusted, data-driven journalism and subscriber-only insights. Want to sponsor this newsletter? [Get in touch here](. You received this message because you are subscribed to Bloomberg's Tech Daily newsletter. If a friend forwarded you this message, [sign up here]( to get it in your inbox. [Unsubscribe]( [Bloomberg.com]( [Contact Us]( Bloomberg L.P. 731 Lexington Avenue, New York, NY 10022 [Ads Powered By Liveintent]( [Ad Choices](