What is up with the Copilots? Two years on from ChatGPT, Generative AI is clearly here to stay. Yet even the most well-resourced companies are struggling to ship cohesive assistants embedded into daily office apps. In an effort to really understand these, I’ve subscribed, cancelled and then re-subscribed (x3!) to paid Microsoft Copilot and Google Gemini plans this year. That’s three times each, in the hope that functionality is starting to match the promise. Spoiler alert: it is not, and with Microsoft in particular beginning to walk back the $33 Copilot Pro subscription here in ANZ, it’s clear that integrating GenAI is, shall we say, difficult. At this point, the best way - and value - to leverage GenAI at work is still natively; through chatting to an LLM in a separate window. Some notes: Presentations: Very Disappointing This is odd. I’m asked weekly for AI-powered presentations, but the products just aren’t there. Copilot in Powerpoint is particularly frustrating. For example, my "create a presentation from [file]" feature simply doesn't work. Forums reveal it's not just me, and it's mind-boggling that such a heavily marketed feature would launch in this state. Gemini isn't much better. While it can technically create slides, the results aren’t blowing my socks off. It can sort of pull from Google Drive files, but the output is "extremely mediocre" (I'm not proud of it, but this is an actual note that I recorded). Email: Looking Good Email integrations shows genuine promise, particularly in Gmail, where: - The side panel adds value for quick exchanges - Real-time draft assessment with thread context works well - Practical features like currency conversion are genuinely useful - The new side panel design is a massive improvement Documents: Getting There Not too many notes. This is improving. Gemini has the edge for me, with increasing linkages to files housed across your drive. I’ve found Copilot in Word surprisingly difficult; for example when marking up contracts. Spreadsheets: Missing in Action Excel is getting there with python integration moving out of beta (or ‘Preview’), but queries are still unwieldy. Google Sheets simply has a long way to go. Overall Status: Jarring Buying these products is surprisingly hard. Neither company is immune (I’ve lost track of Gemini rebrands), but accessing Copilot stands out. For example: - Microsoft's various Copilot plans are poorly explained. I shouldn't have to trawl through T&Cs for whether features apply to individuals vs. Enterprise. - As a result, when basic features like PDF uploads are mysteriously absent or go AWOL, I simply feel gaslit. Is it my fault, for doing it wrong? What's particularly jarring is how all this impacts users. Showing sleek, intelligent tools on a landing page is one thing. In reality, professionals are often paying for subpar, experimental AI features. Things will get there, but shouldn’t be exaggerated until they do. For now, there’s always Claude.
Tim Sharp great breakdown of your experience. We had exactly the same experience in our business. We even tried and I mean really tried to build in co-pilot studio. It was a terrible experience. The simple stuff didn’t work which was baffling as we build in the OpenAI playground and it’s 1,000 better but both ultimately run off the same LLM!
Love this. I fell hook, line, and sinker for co-pilot. I think I unsubscribed almost immediately when it didn't add additional value to chat GPT. If you want a real laugh, you should do some testing on the AI ad creatives—I'm certain 99% never leave the download folder.
Thanks Tim Sharp having recently participated in the AI for Business run by Section and Spark New Zealand I came to similar conclusions. There is still work to be done with the integration assistants, but I'm sure it will get there over time.
Thanks for the summary, very good to get a read out and save some time demo’ing this stuff! Will stick to the native versions for now. Cheers
Helping government and industry understand, navigate and leverage AI.
2dHave you used Microsoft Copilot Studio much and have any thoughts on this?