During natural vision, semantic novelty modulates fixation-related processing in primate cortex
PMC13015500
· 10.64898/2026.03.18.712708
Gap Declaration
So why use this method of measuring novelty? Here, we point out the initial motivation: explore whether self-supervised tuning mechanisms that are effective in computer vision may play a role in biological vision. Future work may explore if intermediate representations of candidate deep-network better match population activity observed at particular stages of the primate visual processing streams, and linking timing to various levels of processing. There is extensive modeling work on static images during a single fixation. In our view, the current results provide tight constraints on future computational models of free-viewing of dynamic visual stimuli.
Abstract
We sample visual scenes with short gaze fixations separated by saccades. While low-level transsaccadic integration is known, semantic integration across multiple fixations remains unclear. We hypothesized that the brain predicts semantic information from one fixation to the next, and therefore postulated a neural signal associated with semantic novelty for each saccade. Novelty was measured using a deep network on foveal vision. Novelty modulated frontal and occipital fixation-related potentials in human EEG during natural viewing of full-length movies (3.4x106 saccades). Intracranial recordings in humans (9.0x104 saccades) and non-human primates (3.3x104 saccades) revealed broadband high-frequency activity modulations in ventromedial/dorsal visual streams and frontal brain areas. This mod…
Conclusions / Discussion
Discussion Our study demonstrates that semantic novelty enhances scalp EEG fixation-locked potentials and broadly enhances iEEG BHA activity, particularly in scene-selective ventromedial visual regions. This spatial profile persists between image and movie viewing. Frontal areas, including frontal eye fields, show early modulation near fixation onset, suggesting anticipation of novelty, while occipital modulation occurs later. These findings, replicated in NHPs, indicate the visual system extracts foveal semantic information and integrates it across saccades to evaluate novelty, providing key spatio-temporal constraints for trans-saccadic vision models. Similarities to known EEG evoked response components Fixed-gaze ERPs often parallel free-viewing fixation-related responses. Our data reveal modulations mirroring established novelty and prediction components during fixed gaze: The mismatch negativity (MMN), a response to stimulus change that occurs even without attention; the P3a, a fronto-central response reflecting automatic orienting to novel stimuli; and the N400, a response to semantic incongruity shown in a range of senses and domains with varying spatial distributions. We ob…
Keeper Review
The Appreciated Gateway must be evaluated by a human keeper.
Does this declaration represent a genuine open research gap?
Does this declaration represent a genuine open research gap?
Structural Hole
65% bridge
Technique originates in computer science; functional analogues in psychology, criminal justice literature are absent.
○ NAUGHT — Open Opportunity
No paper has claimed this gap. Appreciate the opportunity.
Provenance