A retinotopic code structures the interaction between perception and memory systems.

We encode the visual world retinotopically, imposing a spatial code on visual information processing 1-4 . However, models of brain organization generally assume that retinotopic coding is replaced by abstract, amodal coding as information propagates through the visual hierarchy 5,6 towards memory structures 7-12 . This view raises a puzzle for constructive accounts of visual memory: how can mnemonic and visual information interact effectively in the brain if they are represented using fundamentally different neural codes 13 ? Recent work has suggested that even high-level cortical areas, including the default mode network, exhibit retinotopic coding: they contain visually evoked population receptive fields (pRFs) with inverted response amplitudes 14,15 . But, the functional relevance of this retinotopic coding at the cortical apex remains unclear. Here, we report that retinotopic coding at the cortical apex structures interactions between mnemonic and perceptual areas in the brain. Using fine-grained, individual participant functional magnetic resonance imaging (fMRI), we show that just beyond the anterior edge of category-selective visual cortex, category-selective memory areas exhibit a robust, inverted retinotopic code. The negative and positive pRF populations in mnemonic and perceptual areas, respectively, have closely matched visual field representations, reflecting their tight functional coupling. Moreover, the +/- pRFs in perceptual and mnemonic cortex exhibit spatially-specific opponent responses during both bottom-up visual processing and top-down memory recall, suggesting that these areas are interlocked in a mutually-inhibitory dynamic. This spatially-specific opponency further generalizes to familiar scene perception, a task that requires mnemonic-perceptual interplay. Together, these results show that retinotopic coding structures interactions between perceptual and mnemonic systems in the brain, thereby scaffolding their dynamic interaction.