When exploring human cognition and the mechanics of the mind, platforms ranging from collaborative encyclopedias like Wikipedia to analytical commentaries in Psychology Today often highlight how rapid breakthroughs in cognitive psychology reshape our understanding of neural representation. For decades, neuroscientists have mapped how our brains process the external world. But what happens when we turn inward—when we conjure a vivid scene in our mind's eye that has nothing to do with our immediate surroundings?
A team of researchers at Osaka University has recently bridged this gap. Published in Communications Biology, their work demonstrates that the semantic meaning of what a person imagines can be decoded straight from brain wave patterns, even when those mental pictures actively conflict with what the individual is looking at in real life. It is a striking reminder that internal mental states carry rich, distinct signatures that can now be intercepted and understood.
The Architecture of Mental Imagery and Cognition
Ordinarily, our sensory systems are flooded with external data. Whether you are chatting with a colleague, watching a sprawling cityscape, or catching the fading light of a sunset, your brain absorbs visual input that leaves clear electrical traces. Scientists detect these signatures using electrocorticography (ECoG), a clinical technique that records electrical activity directly from the surface of the cerebral cortex.
Yet these neural representations are far from static. They shift continuously based on where we direct our attention and what we picture internally — a dynamic that fits closely with ideas explored in research on convergent predictive processing in the human brain and AI. Lead author Ryohei Fukuma notes that while attention is known to modulate perceived images, a critical question remained unanswered: could voluntary mental imagery alter neural representations when imagined and perceived images semantically conflicted with one another?
To find out, the research team designed an inventive experiment. They partnered with seventeen epilepsy patients who already had subdural cortical electrodes implanted in their brains for clinical evaluation of their seizures. These electrodes provided exceptionally clear intracranial ECoG readouts related to visual perception, offering a rare window into human brain function.
Decoding Brain Waves Through Closed-Loop Feedback
Instead of merely observing passive brain activity, the researchers built a sophisticated closed-loop system that mapped real-time ECoG signals into a visual-semantic space. This builds on a broader effort to read imagined content from neural signals, which recently included decoding imagined musical melodies from invasive neural recordings. In the Osaka study, patients sat before a computer monitor displaying an image driven by their own brain wave decoder. They were then instructed to mentally picture specific semantic categories—such as a landscape, a human face, or a word—while looking at entirely different visual stimuli presented on the screen.
The setup allowed patients to attempt voluntary control over the readout images. By actively imagining a face or a landscape, they could nudge the decoder's output toward their intended semantic category, bending the system to their imagination despite the conflicting visual input flooding their retinas.
Senior author Takufumi Yanagisawa explains that the ECoG readouts generated by imagined images were distinct from those provoked by actual viewed images. More importantly, those neural signatures could be modulated to become even more distinct when patients received real-time feedback from the closed-loop system. The successful control of these feedback images demonstrated that the semantic vector inferred from electrocorticograms became noticeably closer to the vector of the imagined category, even while watching images from entirely different categories.
Interestingly, the time required to establish a sharp distinction between viewed and imagined images varied depending on the semantic category. For instance, generating a clear neural separation for a "word" versus a "landscape" took different trajectories, which likely reflects the distinct cortical networks recruited for processing different types of conceptual information. Furthermore, the modulation of inferred vectors by mental imagery depended asymmetrically on the perceived and imagined categories, highlighting the intricate dynamics shared between perception and imagination.
Clinical Horizons and Assistive Communication
Beyond the fascinating neuroscientific insights into how perception and imagination interact, this technology holds profound promise for clinical applications. Severe neurological conditions—such as amyotrophic lateral sclerosis (ALS) or advanced brainstem strokes—can leave patients entirely locked-in, unable to speak or move while remaining fully conscious. Detecting that hidden awareness is itself an active research frontier, as illustrated by work showing portable brain-computer interfaces doubling covert consciousness detection.
Many existing brain-computer interfaces rely on motor control signals to operate assistive communication devices. However, motor cortical activity often degrades more rapidly as these neurodegenerative conditions progress, limiting the long-term utility of motor-based interfaces. In contrast, visual cortical areas and imagery-based pathways tend to remain robust for much longer periods.
Because this new decoding approach relies on the subject's internal imagery rather than physical movement, it paves the way for a new generation of non-motor communication devices. A paralyzed patient could theoretically communicate complex thoughts simply by picturing specific categories or symbols, translating the rich content of their mind's eye into actionable signals for the outside world.
As research in this domain progresses, the boundary between private internal experience and external readability continues to blur. What started as an academic inquiry into conflicting visual stimulation has opened a promising avenue toward restoring voice and agency to those who need it most.
