Abstract
Understanding neural computations of vision require studying how different brain regions interact with one another. However, functional connectivity across brain regions is often computed as stationary maps, concealing the rich neural dynamics that change at a finer timescale. To better understand how functional connectivity evolves over time, we propose a multimodal framework combining data from intracranial-EEG (iEEG) and fMRI. We recorded iEEG data from early visual (V1/V2) electrodes in 4 patients. Each patient was shown a subset of 1000 stimuli from the NSD-fMRI dataset. Electrodes with significant broadband (70-170 Hz) power increases w.r.t the baseline were considered for further analysis. From the NSD-fMRI dataset, we obtained average fMRI beta-weights for the 1000 stimuli that were repeated thrice across the 8 subjects. Next, for each iEEG electrode we computed a Pearson correlation map with all the fMRI vertices, across the 1000 stimuli, giving us a time x vertices correlation matrix. This provided us with a brain-wide temporally evolving correlation map for each electrode. In all 4 subjects, we observed that the iEEG broadband significantly correlates with the fMRI beta-weights in V1, and with V2/V3 about 5-10 ms later, followed by the ventral temporal regions around 170 ms. Other parietal and frontal brain regions also showed significant correlations after 100 ms. Further, we also observed that these correlations reduce around 450 ms, even though the stimuli were presented for 800 ms. These temporally resolved correlation maps show that V1 representations are not stationary but share representations with higher order visual areas over time. These results may suggest that connectivity to V1 evolves over time revealing feedback inputs from higher order ventral areas around 100-170 ms. Overall, we propose that our multimodal framework enables us to compute functional connectivity at high spatiotemporal resolution reflecting the rich dynamics of interaction across different brain regions.