Microsoft is testing a new feature on Copilot, and this time, it’s coming to PC. Copilot Vision, recently made available on mobile, is now available for testing on PC, and the capabilities are both awe-inspiring and a little scary. Microsoft is also giving file search an AI boost.
Copilot Vision will allow the app to see everything on your screen — provided you’ll want it to. You can share browser windows, your entire desktop, or even a game to get real-time feedback from Copilot. Upon launch, you’ll choose which browser window or app you want to share with Copilot, and from then on, you’ll be able to speak to Copilot the way you would to a tech support agent. You can stop sharing at any given time.
Microsoft shared a screenshot of the feature, showing a Word file being shared with the app. You can talk to Copilot Vision by voice or by text, and the app is said to guide you through the process of whatever you’re trying to do.
The Verge showed off a demo of Copilot Vision that was made available for testing during Microsoft’s 50th anniversary party. You could see Copilot respond to questions during a game of Minecraft, giving the user advice on what to do with certain items. It also guided the user through video editing in Clipchamp, complete with highlighting something on their screen. That last part, the highlighting, isn’t coming in this beta build yet, but it will be available eventually.
Microsoft is also making file search more natural in Windows. The Copilot app will soon be able to help you find files on your PC. You can ask questions such as “Can you find my resume”, and Copilot will know what to look for. Fortunately, you can adjust the permissions for what Copilot is and isn’t allowed to see.
According to the announcement on Microsoft’s Windows blog, the features are rolling out to Windows Insiders in the United States. This means that we won’t all be able to try them out, and it might take a while for Copilot Vision to be rolled out to a broader audience.
