Trying out Elevenlabs “Audio Native” to read the content on the blog, feel free to try it out in the media player above^.
Copilot chat
Welcome back to my Copilot for Microsoft 365 blog series! Feel free to take a look at the first post if you haven’t already.
Today we’ll be looking into the, in my opinion, biggest feature of Copilot for M365; Copilot chat.
It’s actually named “Copilot” but that just creates confusion when talking about all other Copilots, so I usually use “Copilot chat” since it’s a chatbot.
I have also published a video about Copilot chat on my Youtube:
AI chatbots: A Comparison
Before going into the details let’s have a look on the relevance compared to other open models available. You have probably heard of and tried ChatGPT by OpenAI, Claude by Anthropic or other chatbots. These are open chatbots available to the public built upon their Large Language Models (LLM) and are looked at as the “best in class” right now.
I use them both myself and have no problem suggesting others to use them as well, but only for information that you have no problem that might be available to the public at any given time. If you use personal or business sensitive information in these chatbots they can be stored, shared and be used to train the LLM further. This is where Copilot comes into play.
Copilot chat: Free vs Licensed version
The free version
Good news – if you have an Entra ID account, you probably already have access to a free version of Copilot. Just head over to https://copilot.microsoft.com/ or https://chat.bing.com/ and log in with your work account. This version comes with “Commercial Data Protection” by default, so you can use it safely for work-related conversations.
Security and Privacy in the free version
Unlike public AI chatbots, Copilot offers enhanced security for your sensitive information:
- Commercial Data Protection: Available in the free version.
- Encryption: All chat data is encrypted in transit and at rest.
- Data Deletion: Prompts and responses are discarded after sessions end.
- EU Considerations: Data may be processed outside the EU in the free version, but it’s still deleted afterward.
Copilot is, at the time of writing, using OpenAI’s GPT-4 and/or GPT-4 Turbo models when giving answers based on the availability. It’s also using Dall-E 3 for creating images. Copilot will most likely update to GPT-4o and newer models after some time.
The licensed version
If you are lucky enough to get a Copilot for Microsoft 365 license you can also use Copilot chat to access your business data in Microsoft 365. When you ask questions it will use M365 Graph to find information to you across your email, calendar, Teams, SharePoint and OneDrive – everything that you have access to. You can also turn on plug-ins to access data outside M365 – for example the web.
A quick overview of how this looks like:
Understanding RAG (Retrieval-Augmented Generation)
When you use ChatGPT, Claude or any other public available tool on the internet they are trained on a lot of data and that’s their knowledge when you ask them questions. Some chatbots, like ChatGPT, can also access internet to check for information to blend into their answer. But most chatbots are not good at telling you their source of information, so how can you double check that the info you get is correct?
One of the best things about Copilot is that it uses something called RAG. Let’s break it down in an easy way:
When you ask Copilot a question, it doesn’t just rely on its pre-trained knowledge. Instead, it actively searches through your Microsoft 365 content (emails, files, meetings, chats, calendars, and contacts) + plug-ins if you have them enabled to retrieve relevant information. Then, it combines this specific info with its general knowledge from the LLM to generate an answer.
The best part? Copilot gives you footnotes and lists all the sources it used. This means you can easily double-check if the information came from the right places and dig deeper if you need to.
Here’s an example of an easy prompt to get quick up to date on the latest emails and Teams chats. As you can see it gives me information from 7 emails and 10 teams chats + follow-up points for all of these, so in this case it has used RAG to retrieve this information and generate an answer for me.
“Summarize all the emails and teams chats in the last 7 days. I want as many points and information as possible, don’t leave anything out. Give me follow up points based on this.”
In each of the points listed we get a footnote noting which references it has gathered information from and at the end we get all the 17 references listed. When using sensitivity labels (information protection) you will also be told which sensitivity label is used on your referenced files (strongest label will be shown).
Copilot chat across different platforms
One thing to keep in mind – Copilot chat works a bit differently depending on where you’re using it. In my experience, it’s most up-to-date inside the Teams app. That’s where you’ll find access to plug-ins (for now) and where new updates tend to hit first.
For example, a few weeks ago, Microsoft increased the limit on how many files Copilot chat can reference. This update rolled out to Teams, the Outlook app, and microsoft365.com/chat first, before making its way to copilot.microsoft.com and chat.bing.com.
Copilot chat has also started appearing inside other M365 applications like Word, PowerPoint, and OneNote since June. So it’s quite a lot different places you can find it, but just be aware that it might work a bit different in some places.
I made this comparison to make it easy to spot the differences, but be aware that this info might be outdated at any time.
Plug-ins are planned to roll out in December for copilot.microsoft.com and hopefully there won’t be a big difference in how Copilot chat works in the different places after this.
My take on Copilot chat
After using Copilot chat since August last year, I’ve got to say – it’s improved a lot and I find new use cases every now and then. The ability to quickly get a quick overview of all my recent emails and Teams chats is probably one of my most used prompts and it saves me a lot of time searching while helping me remember to follow-up questions from others.
One tip I’ve found useful: Use Copilot (or ChatGPT) to help you create even better and more advanced generic prompts that you can use every day, every week or so. The short and small prompts works fine, but you can fine tune how you want your answers by creating more advanced prompts.
What’s Next for Copilot chat?
Microsoft is constantly updating and improving Copilot and you can follow all planned updates on the roadmap here. One thing I’m really looking forward to is to save our prompts in Copilot Lab and of course updating to newer models from OpenAI.
What about you? Have you had a chance to use Copilot chat yet? Feel free to leave a comment or a question!
Thanks for reading, and stay tuned for more posts in this series!