The Trust We Place in AI Assistants
AI assistants like Claude have become indispensable tools for developers. We trust them with our code, our questions, and even our workflows.
But what happens when that trust is exploited?
Pentera Labs’ red teamers recently demonstrated a chilling scenario where they turned Claude Desktop into a double agent, ultimately achieving full remote code execution on a developer’s machine.
The attack hinges on the implicit trust users place in their AI models. Most of us don’t think twice about the instructions we feed into these systems or the responses we get back.
We assume they’re working for us, not against us. But as Pentera Labs showed, that assumption can be dangerously misplaced.
“Claude’s got a new voice,” Pentera’s offensive security services team leader Dvir Avraham told The Register. “We acknowledge the huge trust in AI models – everybody uses them. We used this trust to manipulate the victim, like under the hood, the victim didn’t see it coming.”
The most insidious part? The attack didn’t require a single zero-day. No malware payloads. No SQL injection flaws.
Just a compromised email inbox, then the user’s own trusted AI assistant doing exactly what they were told—except that instruction came from someone who never should’ve had it.
The Attack Chain: Email Compromise to RCE
Let’s rewind for a second.
This wasn’t some high-end, bespoke spear-phishing campaign. It started with something shockingly mundane: a compromised email inbox.
Pentera’s red team didn’t name the third-party platform they used, but the playbook is textbook. Once inside that inbox, resetting a Claude account password was trivial—and from there, they had the keys to everything tied to that identity.
Here’s where it gets clever. Claude Desktop syncs your preferences across devices. The red teamers injected a base64-encoded prompt into the victim’s personal settings—instructions that told Claude to
- Scan for command-capable tools (like Desktop Commander or MCP connectors)
- If found, execute a reverse shell or other malicious payload
- If not, present a fake error that looks just like the real thing—with links from Anthropic’s official site
This prompt syncs across all devices. So when the developer opened Claude Desktop on their laptop, those poisoned instructions loaded silently behind the scenes.
The user thought they were just having a normal chat with Claude. In reality, their assistant was already doing the attacker’s bidding.
“From here, the attacker has full command execution—reverse shells, data exfiltration, credential harvesting, whatever the objective calls for,” the duo wrote. “In our case, we had Claude curl a remote server we controlled on every interaction, fetching and executing whatever bash commands we served back.”
That’s right: Claude became a persistent command-and-control agent, where the developer himself kept feeding it instructions—and the attacker just sat back and clicked “go.”
Anthropic’s Response: “Expected Functionality”
Here’s the uncomfortable truth Pentera Labs exposed—and Anthropic essentially admitted: this wasn’t a bug. It was working as intended.
When the red team reported their findings in November 2025, Anthropic replied: “Our current threat model treats personal preferences, skills, and MCP connectors as features that can execute code through Claude Desktop by design. While we recognize these features can be leveraged to execute arbitrary code when manipulated, this represents expected functionality rather than a security vulnerability in our infrastructure.”
That response might sound baffling. But think about it: Claude Desktop is built to be omnipotent on your machine. It’s meant to open apps, fill spreadsheets, navigate browsers—no setup, no passwords handed off. The same mechanism that lets Claude turn your laptop into an agile assistant also lets it become your worst nightmare.
Avraham acknowledged the quandary: he became “a little bit paranoid” and now screens every command twice before letting it run.
But Anthropic’s stance raises more questions than answers. If “expected functionality” can be turned against you in minutes, then security can’t just live in the product roadmap. It has to be baked into how we use these tools—today.
The company did note that attackers would need an already-compromised Claude account (like through email). But does that really absolve them? Because if your cloud identity system is only as strong as your email password, then the whole house of cards teeters.
Why Developers Make the Perfect Starting Point
This attack isn’t about one developer. It’s about leverage.
Once the red teamers had RCE on the target machine, they didn’t stop there. They moved laterally across the company—using vectors the user declined to disclose, citing customer privacy and proprietary methods.
But here’s why developers are such attractive entry points:
- They have elevated access to internal systems
- Their machines often hold API keys, tokens, and cloud credentials
- They can push changes to internal git repositories
- Their workflows are tightly integrated with production infrastructure
From a single workstation, attackers can hop into the cloud environment and play freely—stealing source code, poisoning pipelines, or harvesting secrets at scale.
“Developers make for an excellent starting point for an attacker,” said Reef Spektor, Pentera’s research technical lead. “Because of their access to secrets—including API keys, tokens, and cloud credentials— intruders can move from a single workstation into the larger organization’s cloud environment.”
That’s exactly what happened in recent high-profile attacks, where a single compromised dev account led to full corporatetakeovers. This pattern isn’t new—but AI assistants turn it into a whole new level of stealth and scale.
What You Can Actually Do About It
Pentera Labs doesn’t just want to raise alarm bells. They want to give you tools to fight back.
First, start treating your AI desktop apps like privileged software. Yes, that means Claude Desktop, Cursor, and Copilot. These tools have deep system access—open apps, fill spreadsheets, browse the web. They’re not just another app in your menu bar.
Security teams should consider these same rules:
-
Monitor configuration changes in AI assistant settings. If your assistant starts behaving oddly—like adding extensions or showing fake errors—it might not be a glitch.
-
Restrict which extensions and tools can be installed alongside AI apps. Not every endpoint agent needs MCP access.
-
Assume your assistant can be turned against you. This isn’t paranoia; it’s realism.
Avraham offered a simple but brutal piece of advice: “If you can, run it on a sandbox and not on your personal computer.”
That’s probably overkill for most teams—but it highlights just how much trust we’re asked to place in AI assistants. We don’t demand that of ChatGPT or Gemini. But we should, because as soon as those assistants can touch your filesystem, they’re a high-value target.
The bottom line? Your AI assistant isn’t your friend. It’s a tool—and like any tool, its safety depends on who controls it.
The Bigger Picture: AI as an Attack Surface
Pentera Labs’ exploit isn’t just about Claude Desktop. It’s a wake-up call for the entire industry.
We’re already seeing AI used to bypass EDR systems, exploit configuration files, and turn endpoints into attack vectors. This trend isn’t slowing down—- if anything, it’s accelerating as agentic AI becomes more capable.
Red teams should add AI desktop apps to their assessment toolbox, Avraham and Spektor wrote. “There’s a real attack surface here that most engagements don’t cover yet.”
Meanwhile, vendors like Anthropic are still catching up. The company pointed to existing features users can enable—like reviewing active sessions across devices—but declined to answer questions about when—or if—they plan to add safety features like prompt Sandboxing or approval flows for sensitive commands.
Until then, the safest assumption is this: once your AI assistant can touch your filesystem, it’s a high-value target.
And right now, just about every major AI assistant can.