mayaskYou might as well start treating your web browser as if you’re always at work, at least if you want to make use of Microsoft’s brand-new Copilot Vision feature. According to Microsoft’s Executive Vice President Yusuf Mehdi, this feature, which is natively integrated into Microsoft Edge, has the ability to “see what you see and hear what you hear” as you navigate your browser.
All this AI snooping isn’t for nothing. Copilot Vision examines what you’re doing online to provide answers, offer recommendations, and summarize content. For instance, it can work in conjunction with the new Copilot Voice feature. Microsoft demonstrated its capabilities on Rotten Tomatoes, showing a user chatting with Copilot while browsing the website and looking for movie recommendations. Eventually, Copilot settled on an Australian comedy for the Australian speaker, stating that the choice was made because, “well, you’re Australian.” I guess that’s taking personal context into account.
Recommended Videos
Understandably, the idea of an AI looking at everything you do online might not sit well with some people. Microsoft claims that the feature is only available upon opting in, meaning you have to manually enable it, and there will be an indicator onscreen indicating that Copilot is observing what you’re doing. Microsoft also asserts that data from your session will be deleted once you close the session, and that the data won’t be used for training.
In addition, Microsoft says that Copilot Vision will initially only work with a limited number of websites. We don’t have the full list yet, but at the very least, the feature won’t operate on “paywalled or sensitive content.” Microsoft further points out that the feature doesn’t process anything on a webpage; it simply sees the text and images on a page and analyzes them.
Copilot Voice functions in conjunction with Copilot Vision, but it can also operate independently. Across desktop and mobile devices, Microsoft says you’ll now be able to have a conversation with Copilot using only your voice.
Building upon the new voice capabilities, Microsoft is introducing Copilot Daily. It’s almost like a personalized morning news show, gathering stories and providing you with daily details such as traffic and weather. This too is created with a generative AI voice that guides you through your morning news every day. It’s generated based on the content you interact with.
Copilot Voice and Copilot Daily are now rolling out to Windows users in the US and the UK. Voice is also available in Australia, Canada, and New Zealand, but only in English. Microsoft says it will bring the feature to more countries and languages in the near future.
Copilot Vision is set to begin rolling out “soon” in the US, but only for Copilot Pro subscribers. The feature is part of Microsoft’s new Copilot Labs program, which offers Copilot Pro subscribers early access to Microsoft’s latest AI features that are still in the testing phase.
