Abstract
The visual system can effortlessly parse a range of visual information from rich, dynamic input. Most visual neuroscience studies, however, have focused on responses to static images devoid of temporal context. Here we use intracranial electrocorticography (ECoG) recordings from patients watching commercial movies to interrogate the brain's response to more naturalistic visual stimuli. Using commercial movies as stimuli presents many advantages. First, they contain visual information that more closely mimics humans' day-to-day visual experience. Second, and perhaps most importantly, movies contain rich social narratives that provide information at many different levels, from low level visual (e.g. large changes in contrast and illumination when the camera cuts between views) to higher-level social dimensions (e.g. faces, emotions, and social interactions). Although these movies are shown only once to each subject, we can leverage the fact that each movie contains multiple scenes in which similar visual input (e.g. same faces, scenes, etc.) occurs, and use these for repeated measures. Through this analysis we find that the physiological responses at many electrodes, particularly in occipital and temporal regions, are strongly modulated by camera cuts. Despite the dynamic and heterogeneous nature of the stimuli in every cut, several electrodes show a vigorous and reliable response in the majority of cuts. Further, our initial results suggest that these responses are modulated by the content of each sequence of frames, as evidenced by our ability to decode the presence or absence of faces in single cuts. This work shows that these movies, despite inhomogeneity between cuts and lack of repetitions, provide a rich stimulus set for examining visual phenomena. These results provide data and a framework for investigating the neural basis of higher-level visual processing.
Meeting abstract presented at VSS 2016