The Short Answer
Caveman prompting works. Kind of.
JetBrains just published the kind of study every AI-coding team needs to read before rewriting their agent instructions. The headline takeaway? Yes, talking like a grumpy cave-dweller to your coding assistant does shave tokens off the bill. No, it doesn't save you 65% like the viral posts claimed.
The real number sits at roughly 8.5%. That's not nothing, but it's a far cry from the savings posters that swept through developer Twitter and LinkedIn last month.
Where the 65% Claim Came From
The Caveman project started as an open-source effort to strip conversational padding from AI coding agent outputs. The pitch was seductive in its simplicity: if your agent stops saying "I'll help you with that" and just says "ok," you save tokens. Fewer tokens mean lower inference costs at scale.
Early evaluations of just 10 tasks showed savings around 30%. That's already a meaningful number, and it was enough to make the technique go viral. Developers started pasting caveman system prompts into their Claude Code sessions, GitHub Copilot configurations, and whatever other agent frameworks they were running.
But 10 tasks isn't a production workload. It's a warmup lap.
How JetBrains Actually Tested This
Here's where the study gets interesting. JetBrains didn't just run a few quick benchmarks and call it a day.
They used the Harbor open-source evaluation framework paired with tasks from SkillsBench — 86 real-world software engineering tasks run through Claude Code. Paired comparisons, same tasks, one with caveman prompting and one without. That's the kind of methodology that actually means something.
The engineer behind the work, Denis Shiryaev, wrote up the findings in a blog post that's worth reading in full. The pattern was clear: as the test expanded beyond those initial 10 tasks, the savings dropped sharply. From 30% down to roughly 8.5% across the full benchmark.
That's not a failure of the technique. It's a lesson in how token economics actually work inside agentic coding workflows.
Why the Savings Are So Modest
Here's what most people miss when they talk about token savings: the conversational layer is a small slice of the pie.
Modern coding agents spend the vast majority of their tokens on things that have nothing to do with how politely they phrase a response. Reading project files. Reasoning through tasks. Invoking tools. Generating actual code.
Trim the "sure thing, let me take a look at that" down to just "ok," and you've saved maybe five tokens per exchange. Meanwhile, the agent is burning hundreds of tokens just reading your codebase and another hundred generating the function you asked for.
The caveman approach targets the wrong bottleneck. It's like trying to save money on your electricity bill by switching to a cheaper lightbulb while leaving the HVAC running at 72 degrees.
This isn't a knock on caveman prompting itself. It's just that the savings it delivers are proportional to what it actually changes — and that's a relatively small portion of total token consumption.
The Cost Anomaly That Almost Ruined Everything
Here's a detail that almost gets buried but matters more than you'd think.
JetBrains found that on a single dependency-audit task, the caveman run actually cost more than the baseline. Why? That one task pushed Claude Code past its long-context pricing tier. The caveman technique didn't cause the spike — a similar outlier appeared in an earlier baseline run too. It was just the workload, not the prompting style.
This is the kind of thing that makes enterprise cost modeling frustrating. You can't just multiply per-task savings by volume and call it a budget forecast. Context windows, pricing tiers, and outlier tasks all conspire to make the math messier than anyone wants.
For teams actually running AI agents in production, this is the part that keeps you up at night. Not whether caveman saves 8% or 12%, but whether your cost model accounts for the tasks that break the pattern.
The Good News: Nothing Got Worse
Let's be clear about what JetBrains didn't find.
No degradation in task success rates. No drop in code quality. No increase in execution time. The caveman style didn't make the agent dumber, slower, or less reliable.
That's actually significant. A lot of prompt-engineering techniques come with tradeoffs — you save tokens but the output quality tanks, or you get faster responses but more hallucinations. Caveman prompting sidesteps that entirely.
It's a free pass on quality for what turns out to be a modest cost reduction. Not the revolution some promised, but still a net positive if you're running agents at any meaningful scale.
What This Means for Your Team
If you're considering adopting caveman prompting in production, here's the honest take:
Validate it yourself. Don't trust the 65% number. Don't even fully trust the 30% early benchmark. Run your own paired tests against your actual workload — the tasks you care about, in the context windows you actually use.
The technique is safe. It won't break anything. But the savings will likely look more like 8-10% than anywhere near what the viral posts suggested.
For enterprises, this is a reminder that prompt-engineering optimizations should be treated as hypotheses to test, not gospel to adopt. The production workload always tells the real story.