The Surprising Simplicity Behind How Neural Networks Actually Learn
Here's something that might sound counterintuitive at first: your brain doesn't learn by building new pathways from scratch. It learns by getting better at what it already has. A new study from Bar-Ilan University is lending strong support to this idea, showing that both artificial and biological intelligence rely on refining existing connections rather than endlessly expanding their architecture.
The research, led by first author Yanir Harel—a Master of Science student—and Professor Ido Kanter at Bar-Ilan University's Department of Physics and the Gonda (Goldschmied) Multidisciplinary Brain Research Center, examined how neural networks handle language-learning tasks as they grow more sophisticated. Their findings, published in Physica A: Statistical Mechanics and its Applications (DOI: 10.1016/j.physa.2026.131855), suggest a unified computational principle running through both artificial and biological systems. Intelligence, it seems, emerges not from adding more components, but from improving how the ones you've got work together.
How Researchers Tested Learning's True Mechanism
The team trained artificial neural networks on language-learning tasks using progressively larger datasets. As expected, the models got significantly better at learning as the data volume grew. But here's where it gets interesting: even as performance improved, the researchers found that these networks could still lose roughly the same proportion of their synaptic connections without any meaningful drop in functionality.
This was measured through a process called network pruning—essentially removing connections and seeing what happens. The fact that improved learning didn't require more complex networks, or more connections, pointed directly to a single conclusion: learning happens by refining the strength of internal connections (synaptic weights), not by restructuring or expanding the network itself.
"This finding is particularly intriguing in the context of biological brains, where the number of neurons remains approximately constant," Harel noted. "Moreover, there is currently no evidence for a biological mechanism that would enable large-scale, continuous reconfiguration of neural network topology during learning."
Why This Matters for Biological Brains
In a mature biological brain, the total number of neurons stays essentially fixed throughout adult life. There's no known mechanism for large-scale, continuous structural rewiring of overall brain architecture during everyday learning. So how does the brain learn? The answer, supported by decades of neuroscience, lies in synaptic plasticity—the ability of existing connections between neurons to strengthen or weaken over time based on activity.
The Bar-Ilan study aligns neatly with this biological reality. By demonstrating that artificial networks achieve better performance without adding structural complexity, the research bridges a gap between how we build AI and how nature builds minds. Both systems, it appears, converge on the same strategy: optimize cooperation among existing components.
This has real implications for our understanding of cognitive reserve and resilience. If learning truly depends on refining connections rather than building new ones, it may help explain why certain brains show remarkable resistance to degeneration. A richer web of well-tuned connections could provide more robustness against damage, a physical basis for cognitive reserve that doesn't require generating new neurons.
Measuring Learning: CpW and CpL
To quantify what they were observing, the researchers introduced two new metrics. The first, called capacity per weight (CpW), measures the maximal capacity of a feedforward neural network relative to its number of weights. In theory, this could serve as a quantitative measure of natural language processing (NLP) task performance.
Direct simulations of NLP pre-training tasks using deep architectures revealed two important observations: deep networks can be heavily pruned without affecting accuracy per token (APT), and the maximal pruning level that preserves APT is independent of the training dataset size. This last point was unexpected, and it called CpW's relevance to real-world NLP tasks into question.
CpW decreases as a power-law with the number of output units, which doesn't make much sense when you're dealing with tasks that have many possible outputs. To address this limitation, the researchers introduced a second measure: capacity per label (CpL). This metric was designed specifically for classification tasks and was validated using a toy model based on archetype input vectors. The results showed that maximal dilution preserving the accuracy of an undiluted model decreases with the number of output units while remaining largely independent of training dataset size and the number of hidden layers.
Implications for Artificial Intelligence Design
Perhaps the most practical takeaway from this research is what it means for how we build AI. The prevailing approach in machine learning has been to keep scaling up, more layers, more parameters, more compute. The Bar-Ilan findings suggest an alternative path: instead of endlessly expanding network sizes, focus on optimizing how existing weights cooperate and pruning unnecessary connections.
This approach could lead to significantly more compact, energy-efficient, and computationally effective machine learning models. In an era where training large language models consumes enormous amounts of energy and resources, a design philosophy centered on refinement rather than expansion could have profound environmental and economic consequences.
The research was tested using BERT architectures pre-trained on tiny subsets of the Wikipedia dataset and fine-tuned on the FewRel classification task. While the datasets were small, the principles revealed, particularly the independence of maximal pruning levels from training data size, suggest broader applicability.
Bridging Artificial and Biological Intelligence
What makes this study particularly compelling is its convergence across domains. The same computational principle, refinement over expansion, appears to govern both artificial neural networks and biological brains. This isn't just a coincidence; it may reflect a fundamental constraint on efficient learning.
Adding new structural components is expensive, computationally, metabolically, and in terms of developmental time. Refining existing connections, on the other hand, is far more economical. It's no surprise that evolution and engineering, working through entirely different processes, have arrived at similar solutions.
As the researchers put it, "intelligence may emerge less from adding new components and more from improving how existing ones work together." That's a simple idea, but it's one that could reshape how we think about both AI development and the nature of biological cognition itself.
Source: Bar-Ilan University
Lead authors: Yanir Harel (M.Sc. student), Tal Halevi, Ronit D. Gross, Shalom Rosner, Prof. Ido Kanter
Journal: Physica A: Statistical Mechanics and its Applications
DOI: 10.1016/j.physa.2026.131855
Article reference: https://neurosciencenews.com/learning-refining-neural-connections-31158/
