Abstract
Whilst the importance of multimodal information is repeatedly highlighted for self-body sensations, the integration process of the multimodal information for these sensations is still unclear. We examined the contribution made by gaze position to the integration of multimodal information under the hypothesis that visual and haptic information should be within a spatiotemporal window of gaze and spatial attention to achieve the integration. To investigate the critical temporal delay for this window, we measured how visual feedback delay changes eye and hand behaviors and self-body sensations. Participants executed a block copying task that involved manually collecting and arranging colored blocks to duplicate a pattern observed on a delayed video image. As delay increased, the score of the questionnaire on self-body sensations decreased. This decay slope changed after a delay of approximately 350 ms, which was when distribution of fixation duration and hand velocity also showed qualitative change. These results suggested that participants' gaze and hand behaviors changed at this value. When the delay was relatively short, the hand position followed gaze position and thus shared the same movement patterns. However, when the delay was relatively long, this relationship collapsed, suggesting that visual and haptic feedbacks are not within the temporal window for the integration and thus participants relied on their haptic sense more heavily than vision to avoid attentive demand. Our results suggest that the 350 ms visual feedback delay is the critical point for changing the demand needed for multimodal process around gaze position. To examine the contribution of gaze position to multimodal binding or self-body sensations more precisely, we are currently conducting a gaze-contingent window experiment for the real-time video image, in which participants' central vision was replaced by delayed image, to investigate critical size of spatial window and its interaction with temporal window for intuitive operation.
Meeting abstract presented at VSS 2014