5 Features Windows 11’s AI Copilot Must Be Really Useful

AI has become a big part of Microsoft’s strategy over the past year. Of course, that can be said for many companies, as AI seems to be the hottest buzzword right now. But Microsoft recently took the bold step of bringing it to Windows 11 in the form of Windows Copilot, a new AI assistant that is (unofficially) a successor to Cortana. It’s powered by the great GPT-4 language model used n Bing Chat to create a tool that hopes to be more useful than ever.

While the potential is exciting, the current implementation of Copilot in Windows Insider builds is very basic and not that different from simply using Bing Chat on the web. I have a few ideas for what Microsoft can do to make Copilot a really important part of Windows 11.

1 Support all system settings and main functions

The big draw of Windows Copilot at launch was that it could interact with your PC in ways that Bing Chat, as just a browser, really couldn’t. And this is a good starting point. Many power users already like to use text-based interfaces and keyboard commands to perform certain tasks, so being able to change many of these settings from one place has great potential.

However, the current implementation only supports a few features, such as being able to switch to light or dark mode, turn on do not disturb or take a screenshot. This really needs to be expanded upon all the Windows settings that can be found in the Settings app (the Control Panel should probably be left at this point), or at least a lot more than what we have right now. It could change the accent color, disable one of the displays in a multi-monitor setup, change playback devices, and so on. It could even start a focus session.

If Windows Copilot could help with all of this, it could make interacting with certain settings much faster, especially for people who do it often. There’s a lot that can be done, and Microsoft has promised some of it, but it hasn’t delivered yet.

2 Easy app hooks

Another thing I feel will make or break Copilot’s usability is its integration with other apps. In the spirit of becoming a centralized AI assistant, Copilot should be easily integrated with all kinds of third-party apps. We saw this in action when Microsoft demonstrated Copilot at this year’s Build and used it to play music with Spotify. But we haven’t seen that become functional yet, and it has to go beyond Microsoft’s usual partners.

Copilot should be able to open apps and start a specific task in them, or play a specific show on Netflix or anything else that might be useful in those apps.

3 Integration with File Explorer

I know people are probably tired of Microsoft shoehorning in unwanted services wherever it can, but I think Copilot integration into File Explorer could make it useful for more than just users who want to type instructions in Copilot, especially because it requires having the Copilot window that takes up space on your screen almost permanently.

It might be interesting to have a feature in File Explorer where you can right click on a file and select an option called “Send to Copilot” which will prompt it to ask what you want to do with that file. For example, with an image you can ask Copilot to remove the background, or you can ask it to transcribe an audio file. These are both features we’ve seen featured, but would normally require you to drag and drop the file into the Copilot panel. I think being able to access it directly from File Explorer would be welcome (but the ability to turn it off would be nice too).

4 Voice control

I suspect I’m alone on this one, so I’ve pushed this one down a bit. Voice commands are obviously more popular on phones, and right now Bing Chat only supports this feature if you’re on your phone. But I think it would be nice to have voice commands supported on laptops and PCs as well. It can be useful to be able to change all these settings, start playing music or open an app with your voice. And the same goes for asking Bing questions. Cortana supported it when Windows 11 was introduced, and Microsoft wanted it to be a big thing, so I’m not sure if it would actually gain much traction, but I can definitely see uses for it.

With a browser, you have to go out of your way to initiate voice interaction, but with a built-in feature, you could have an activation keyword or shortcut to quickly speak your commands. I imagine it wouldn’t be that hard to implement since voice recognition is already used in several parts of Windows.

5 Screen reading and OCR

One last thing I would love to see Copilot be able to do is read the information on the screen and especially optical character recognition (OCR). Right now you can ask Copilot to summarize a page, but it has to be on a website open in Edge, which greatly limits functionality. While the ideal approach would be to make this work with other browsers natively, it would be interesting if Copilot could basically act as a screen reader (like Narrator) and use it to extract information from any page in any browser or app to sum it up.

Another cool feature would be OCR, or the ability to recognize text in images, so you can ask Copilot to pull text from an image or PDF scan, and simply copy it or summarize it. This one is actually not that crazy as it has been reported that some sort of OCR feature is planned for Windows 12 so I’m sure we could see something along these lines.

There is more untapped potential

I explored my personal preferences and desires here, but with something as capable as Copilot and the great language models that drive it, I’m sure there are many more ideas that people could find useful. Regardless, the future of Copilot and Windows is pretty exciting, provided Microsoft can deliver on the promise of this product. Whether that will actually happen is another question entirely.