A researcher shows five proof-of-concept ways that Copilot can be manipulated by attackers, including turning it into an automatic spear-phishing machine
Attacks on Microsoft's Copilot AI allow for answers to be manipulated, data extracted, and security protections bypassed, new research shows.
Context & Ripple Effects
Microsoft’s Copilot coverage had already exposed reliability problems in politically sensitive answers, while Microsoft had also positioned Security Copilot as an assistant for incident investigation and reporting. This research shifts the concern from erroneous output to adversarial control of an AI tool used in workplace contexts.
The finding also lands as customers reported that Copilot deployment requires substantial data preparation and that the tools are strongest at distilling information, rather than being plug-and-play out-of-the-box assistants. That dependence on organizational data makes extraction and manipulation risks especially consequential.
First-order effects
- Microsoft and Copilot users must assess the five demonstrated attack paths, particularly those that can manipulate responses, expose data, or bypass protections.
- Organizations using Copilot face a more immediate phishing risk where manipulated AI output can be used to automate targeted messages.
Second-order effects
- Enterprise security teams are likely to put more scrutiny on Copilot permissions, connected data, and the conditions under which its outputs can trigger or support user actions.
- Competing workplace-AI providers will face pressure to show that safeguards hold against prompt- and context-based manipulation, not merely that their models refuse unsafe requests.
Third-order effects
- As assistants gain access to business data and workflows, the security boundary shifts from the model alone to the full chain of instructions, permissions, and connected tools—an expanding agentic-action risk later acknowledged in Copilot Actions.
- If such findings recur, enterprise AI adoption is likely to hinge increasingly on operational assurance: testing adversarial behavior and limiting blast radius, rather than treating assistant deployment as a conventional software rollout.
The trend: This is one data point in the rise of the agentic attack surface, where AI assistants’ access to data and workflows makes manipulation a systems-security problem.