Microsoft Adds Voice and Vision Features to Copilot
Latest Version of AI Assistant Is Compatible with iOS, Android, Windows, and WhatsApp
Microsoft is trying to show that its massive investments in artificial intelligence are paying off by adding voice and vision features to its consumer-focused assistant.
The updated Copilot app will read users a news summary in a feature called Copilot Daily, with personalized reminders coming later.
Copilot Vision, which is still in beta, will answer questions about text and images users browse online and offer guidance, such as suggesting ways to furnish an apartment someone is considering buying.
The updated assistant, which began rolling out Tuesday, works on iOS, Android, Windows and web pages, and includes a new version of WhatsApp.
Hazem Abbas
Microsoft’s feature experience
The vision feature requires user consent to turn on, and in beta, the feature is being tested on a limited list of popular websites so Microsoft can collect data and ensure safety.
The company also said that the data will not be stored or used for training purposes, and that the tool will not use paid or “sensitive” content in beta.
While Microsoft has been a pioneer in consumer AI thanks to its investment in OpenAI, competitors like Apple and Google parent Alphabet are now continuing to launch AI services targeting millions of Android and iOS mobile users.
Microsoft lacks a strong presence in this market, so its AI strategy is largely based on Windows laptops and tablets.
Hazem Abbas
Earlier this year, the company partnered with hardware manufacturers to market a line of computers specifically designed to run AI applications. So far, the response to these devices — which are sold under the Copilot+ PC brand — has been mixed.
In an attempt to drum up some excitement, Microsoft has also announced several new AI features specifically for these devices.
A new photo editing tool lets users enhance old or low-quality photos, as well as fill in and delete parts of images to make editing easier.
A plain-English search also lets users find documents and images by describing them rather than having to know the file name, while a Click to Do feature offers a list of suggested steps or tasks based on the task the customer is working on. The new Copilot app will also be available with a single click on Copilot+ PC devices.
Meeting the emotional needs of the user
Microsoft’s head of consumer AI, Mustafa Suleyman, has pledged to go beyond the useful, practical products the company has long been known for. In an interview, he talked about creating new interfaces and building “experiences that are deeply sensitive to meeting the emotional needs” of users.
He acknowledged that getting there will require a shift in thinking at Microsoft, as well as building new business models.
“Now that we have these conversational interfaces that feel engaging—that focus on the emotional intelligence of the experience, not just the utility of the use—it requires us to develop entirely new skills within the organization,” Suleyman added.
In a blog post, Suleyman explained his vision for the future of AI at Microsoft, saying, “With your permission, Copilot will eventually be able to act on your behalf, making life’s complexities easier and giving you more time to focus on the things that matter to you.” Suleyman cited examples like scheduling doctor’s appointments, planning a kid’s birthday party, and “making a tough life decision.”
