Abstract
Recently, we found that visual evoked potentials (VEPs) for natural textures systematically correlate with image features and that texture images can be reconstructed from VEPs based on this correlation (Orima & Motoyoshi, 2021; Wakita et al., 2021). In the present study, we attempted to decode the neural information that supports the perception of materials and properties of various surfaces from VEPs. We measured VEPs for 191 surface images classified into 20 material categories such as rock, fabric, and metal. Images were presented for 500 ms in random order followed by 750 ms blank. Each observer viewed surface images foveally with no other tasks, and EEG signals were recorded from 31 electrodes. Next, we constructed a multimodal variational autoencoder (MVAE) using surface images and corresponding VEPs as input. Using latent variables of the MVAE, we were able to classify the material categories with a 60 % accuracy rate, and to predict the evaluation values of the 13 texture attributes such as glossiness and transparency with high statistical significance. In addition, we confirmed that the MVAE was able to reconstruct surface images with photorealistic quality using only VEP data as input. These results support the idea that rich perceptual impressions, including surface materials and properties, are based on cortical representations of spatially-global image features and can be reflected in low-spatial-resolution EEG patterns.