ProBackend
ai impact on scientific professions
2 hours ago7 min read

When AI Companies Solve Math Problems in Secret, Mathematicians Say Stop

The International Mathematical Union has endorsed a declaration warning that proprietary AI systems and tech industry practices threaten the integrity of mathematical research and the profession itself.

Renee Cho

Here's a thing that should've been obvious but apparently needed to be said out loud: mathematicians are worried about AI companies eating their lunch, and they've decided to do something about it.

The International Mathematical Union just endorsed a formal declaration warning that proprietary AI systems and the practices of big tech companies threaten both the integrity of mathematical research and the profession itself. That's not a gentle nudge toward better behavior. That's a line in the sand drawn by people who spend their lives being precise about things.

What makes this declaration interesting isn't just that it exists. It's what the mathematicians actually flagged as problems. They're not anti-technology. They're not Luddites throwing wrenches at progress. They're pointing at specific, observable behaviors from AI companies and saying: this is how you break trust in a scientific field.

The declaration covers universal principles that virtually every discipline and industry is going to have to grapple with in the coming decades. Math just happened to be the field where these tensions crystallized first, because mathematics has always been one of the most open, collaborative, and verification-driven fields in all of academia. You can't fake a proof. Or at least, you shouldn't be able to.

But AI companies are trying anyway.

The Declaration That Says Stop

The Proprietary Black Box Problem

Let's talk about what actually happened, because the details matter more than the headline.

OpenAI issued a press release about solving a mathematical problem. A real one. The kind of thing that would normally take months, maybe years, for a team of mathematicians to work through. Except here's the catch: they kept the model proprietary. Closed source. Nobody outside the company gets to see how it was done.

Ryuki Ochigame, one of the researchers involved, put it bluntly to The New York Times: "The AI model is proprietary and unavailable to anyone outside the company. We get a flashy promotional video, while basic information needed to assess the scientific meaning of the result is kept secret."

That's the core issue right there. You get a slick video. Maybe some cherry-picked benchmarks. But the actual methodology? The training data? The architecture choices that made this "breakthrough" possible? All locked behind a paywall of corporate secrecy.

This is what happens when you apply "move fast and break things" to academic research. Except the things being broken aren't user interfaces or shipping schedules. They're peer review, transparency, and the basic social contract that lets science actually work.

Think about it from the perspective of a working mathematician. You spend years developing expertise in a specific area. You learn to verify claims, to trace logic chains, to build on the work of others. Then some company releases a press release claiming they've solved something significant, and you can't verify any of it because the model is proprietary. You're supposed to just take their word for it?

That's not how science works. That's how marketing works.

The Proprietary Black Box Problem

When the Machine Gets It Wrong (and Won't Admit It)

Here's another problem that should've been obvious but apparently needed documenting: large language models generate plausible-looking solutions that are often incomplete or wholly incorrect.

I know what you're thinking. "How do we know they're wrong?" Good question. That's exactly the problem. These models produce answers that sound confident, that follow the right format, that use the right terminology. But confidence isn't correctness. And when you can't examine how the model arrived at its answer, you can't distinguish between a genuine solution and a really convincing hallucination.

The declaration specifically calls out how LLMs "misleadingly use specific mathematical tasks as metrics for the general reasoning capacities of commercial products." Let me translate that: companies test their models on a narrow set of math problems, show impressive results on those specific tasks, and then imply their systems have broad reasoning capabilities. That's not science. That's a demo day pitch.

And then there are the press releases. Real-time announcements about "the great mathematical problem we've just solved" without allowing math professionals to see how it was done or fully vet the answers. This isn't just bad form. It's actively harmful to the ecosystem that produces actual mathematical knowledge.

The Erdős unit distance conjecture came up in discussions around this, and it's a perfect example of why this matters. The conjecture involves understanding relationships between points in space, and disproving it requires actual reasoning, not pattern matching. When AI systems claim to have "solved" problems like this through what's essentially sophisticated autocomplete, they're not advancing mathematics. They're confusing the public about what mathematics actually is.

The Privatization of Public Knowledge

The real problem here isn't even human versus machine. It's closed, proprietary models versus the open communication that's so fundamental to academic work.

Mathematics has always been a collaborative enterprise. Mathematicians publish papers, share results, build on each other's work, debate methods, verify proofs. The entire system runs on transparency and accessibility. You can't do mathematics in a vacuum, and you certainly can't do it if the tools you're using are locked behind corporate walls.

When AI companies develop foundation models trained on decades of mathematical research, then refuse to share how those models work or what they've learned, they're effectively privatizing academic knowledge. The research was publicly funded. It was published in open journals. It was built by thousands of mathematicians working across institutions and borders. And now a handful of for-profit companies are using it to build products they have no intention of making transparent.

This isn't hypothetical. This is happening right now. AI companies are trying to get their foundation models to replace humans in higher-order work like research and art, while simultaneously claiming that their "AI" efforts are only for automating menial tasks. Pick a narrative, folks. You can't have it both ways.

The declaration is essentially saying: if you want to use mathematical research to train your models, you need to respect the ecosystem that produced that research. That means transparency. That means allowing verification. That means not treating academic knowledge as free raw material for your proprietary products.

It's a reasonable ask. And the fact that it needs to be made explicit says something troubling about where we are.

For more on how open-source communities are grappling with AI governance and transparency, see our coverage of the risks of agentic AI in open source.

What This Means Beyond Mathematics

Here's where it gets interesting. The IMU declaration isn't just about math. It's about principles that apply to every discipline and industry facing similar pressures from AI companies.

The Pontiff's AI Encyclical reached remarkably similar conclusions, though from a completely different angle. The Church produced a 40,000-word sociological and historical and theological treatise on the topic. The mathematicians went with something more direct and mechanistic. But both arrived at the same place: people matter more than techbro stock valuations.

That's not a trivial statement. That's a fundamental claim about where value resides in society.

The labor market implications are staggering. If AI systems can produce plausible mathematical solutions without actual understanding, what happens to the value of human expertise? Not just in mathematics, but in law, medicine, engineering, all the fields that depend on trained judgment and verified knowledge?

We're already seeing the early stages of this. Legal AI tools that generate convincing but incorrect case analysis. Medical AI systems that produce plausible diagnoses without understanding pathophysiology. Engineering tools that suggest solutions without grasping the full context of constraints and tradeoffs.

The AI industry, from the perspective of investors and executives, is about redirecting business spending toward their proprietary models. If that reduces labor costs, great. If it enables new processes that weren't practical before, awesome. The money just needs to keep flowing.

But here's the thing: if locally-hosted or open models win out in the market, that would be disastrous for the current AI industry structure. They can't capture the trillion-dollar revenues needed to justify current investment levels without massive-scale proprietary models and consumer lock-in. That's the model Silicon Valley has used for the last two decades to dominate software and the internet.

The declaration is a warning shot. It's saying: we see what you're doing, and we're not okay with it.

Whether anyone listens remains to be seen. But at least the warning is out there. And it's well-timed.

For broader context on how policymakers are responding to these challenges, see our analysis of Dario Amodei's policy blueprint for the AI exponential era.

More blogs