Neural correlates of the stereokinetic effect revealed by functional magnetic resonance imaging

Abstract

The stereokinetic effect (SKE) refers to a visual phenomenon in which a two-dimensional figure rotating in the fronto-parallel plane about the visual axis can create the impression of a three-dimensional (3-D) object. Although several characteristics of SKE suggest that the perceptual mechanisms involved in SKE may differ from those of the kinetic depth effect (KDE), the differences between SKE and KDE in neural mechanisms have not yet been investigated. In order to determine the cortical areas involved in SKE, we presented a variety of SKE stimuli in a series of functional magnetic resonance imaging experiments, controlling for motion and contrast energies as well as stimulus presentation paradigm. Cortical activation associated with SKE was observed in the middle temporal complex (hMT+), lateral occipital area (LO), V3B, inferior temporal gyrus (ITG), fusiform gyrus (FG), and dorsal intraparietal sulcus anterior (DIPSA). On the other hand, ITG, FG, and DIPSA were also activated by the static versions of SKE stimuli. hMT+, LO, and V3B are also known to be activated in KDE. These findings suggest that general motion-dependent 3-D object processing may be performed in these areas.

Introduction

Generally, rotary motion in the fronto-parallel plane about the visual axis cannot convey depth information, because it does not produce changes in velocities of object points in a two-dimensional (2-D) image, that is, velocity gradients. Velocity gradients are a very important cue not only for the kinetic depth effect (KDE; Wallach & O'Connell, 1953) but also for other types of 3-D structure-from-motion (SFM; Ullman, 1979). As an exception to this, however, observers can perceive a 3-D object when a specific 2-D figure rotates in the fronto-parallel plane about the visual axis. This remarkable perceptual phenomenon is termed the stereokinetic effect (SKE; Musatti, 1924). 

Compared with other depth effects, the perception in SKE has several interesting characteristics. First, it takes more time to perceive a 3-D object. Although this time can be reduced with practice, even practiced observers require some time to obtain the perception (Bressan & Vallortigara, 1987; Howard & Rogers, 2002). Second, contrary to the strong impression of a 3-D object, depth order remains ambiguous in the absence of other depth cues (Howard & Rogers, 2002). Finally, individual differences are much larger in both perceptual time and impression. These characteristics of SKE suggest that the perceptual mechanisms involved in SKE may differ from those in KDE. Although the relationship between SKE and KDE has been discussed in many psychological and model studies (Caudek & Proffitt, 1993; Proffitt, Rock, Hecht, & Schubert, 1992; Wallach & Centrella, 1990), the differences between them in neural mechanisms remain unclear, since no studies have investigated the cortical areas involved in SKE. 

The aim of the present study was therefore to reveal what roles visual areas play in SKE using functional magnetic resonance imaging (fMRI). Our fMRI-based approach to SKE can contribute not only to revealing its mechanism in more detail but also to clarifying whether the mechanisms of SKE and KDE are similar or different by comparison with the results of a handful of fMRI studies performed on general 3-D SFM including KDE (Kriegeskorte et al., 2003; Murray, Olshausen, & Woods, 2003; Orban, Sunaert, Todd, Van Hecke, & Marchal, 1999; Paradis et al., 2000; Peuskens et al., 2004; Sereno, Trinath, Augath, & Logothetis, 2002; Vanduffel et al., 2002). For example, if the emergence of SKE is associated with activation of cortical areas other than those involved in KDE, this will indicate that SKE is processed via mechanisms different from those of KDE in these areas. In contrast, if it is associated with activation of the same areas as KDE, this will indicate that these areas play vital, more general roles in depth perception and that SKE may be a type of 3-D SFM similar to KDE. 

In attempting to identify cortical areas involved in SKE, we controlled for motion and contrast energies as well as stimulus presentation paradigm. It should be noted that identification of cortical areas can be affected by control stimuli in fMRI experiments. Since cortical activity is defined by comparison of a target stimulus with a control stimulus, response selectivity for 3-D SFM depends on which 2-D control stimulus is adopted. For example, if a coherent 3-D SFM stimulus is compared with an incoherent 2-D motion stimulus, effects of motion coherence and the presence of shape may be added to activation associated with 3-D SFM. It is thus desirable to employ a pair of visual stimuli in which motion energy and 2-D structure are equivalent. 

General methods

Subjects

All subjects had normal or corrected-to-normal vision, and none had a history of neurological or psychiatric disease. All procedures were approved by the Ethics Committees of the Graduate School of Medicine and that of the Human and Environmental Studies, Kyoto University, and informed written consent was obtained from each subject prior to participation. 

Visual stimulus presentation

Visual stimuli were created using the C language and a graphics library (Open GL, Silicon Graphics, CA) and were driven by a personal computer (Evo N800w, Compaq, TX). Stimuli were back-projected onto a screen using a digital light processing projector (U2-X2000, Plus, Japan) with a spatial resolution of 1024 × 768 pixels and refresh rate of 60 Hz. Subjects viewed a translucent display screen positioned in the bore of the magnet, at a distance of approximately 20 cm from the eyes, through a mirror angled 45 deg to the line of sight. Subjects were instructed to always fixate on a red fixation point at the center of the screen. The size of the fixation point was 0.25 deg in radius. 

Two types of stimulus presentation paradigm were used. One was a block design with alternating presentations of target and control stimuli without blank intervals ( Figure 1A). The direct comparison of cortical activity between target and control stimuli is useful to identify cortical areas involved in SKE, since cortical activity due to motion of target and control stimuli can be canceled out. Generally, motion stimuli evoke strong cortical activity. Therefore, impact of stimulus onset cannot be avoided, especially when stimuli are presented after the presentation of a uniform gray field serving as baseline. This also applies to SKE stimuli rotating at a rapid rate. It is thus necessary to remove the motion factor by presenting motion stimuli constantly in order to identify cortical areas involved in SKE with precision. The other stimulus presentation paradigm was a block design with blank intervals ( Figures 1B and 1C). 

Figure 1

 

Time courses of three types of experiments. (A) In Experiments 1A and 2A, target (SKE) and control (2-D) stimuli were presented for 16 sec alternately with the 2-sec transition period, but without blank intervals. “T” corresponds to the epoch of a target stimulus presentation, while “C” corresponds to the epoch of a control stimulus presentation. A 16-sec control stimulus was presented first so that the first target stimulus epoch did not follow a presentation of the uniform gray field. The alternating block was replicated six times. The experiments included 2-sec transition periods, in each of which the stimulus of the former epoch was transformed smoothly into that of the latter, and each of which was comprised of the final 1 sec of the former and the first 1 sec of the latter epoch, in order to suppress activation caused by sudden changes of stimuli. (B) In Experiments 1B and 2B, the same target and control stimuli as in Experiment 1A or 2A (indicated as “S”) were presented randomly for 16 sec with 16-sec blank intervals. Each run started from a 30-sec presentation of the uniform gray field. Each motion stimulus was presented four times in each run. (C) In Experiments 1C and 2C, static versions of the target and control stimuli in Experiment 1A or 2A (indicated as “S”) were presented randomly for 8 sec with 16-sec blank intervals. Each run started from a 30-sec presentation of the uniform gray field. Each static stimulus was presented at four different positions (polar angles of 45, 135, 225, and 315 deg from the fixation point) four times in each run for Experiment 1C and eight times in a run for Experiment 2C.

Data acquisition

All functional MR images were acquired on a 3-T whole-body MRI scanner (MAGNETOM Trio, Siemens, Germany) using an 8-channel phased-array head coil. 

In Experiments 1A and 2A, the experimental paradigm was a block design with alternating 16-sec presentations of target and control stimuli ( Figure 1A). Each run started from a 14-sec presentation of a uniform gray field with the fixation point. No image was acquired during this period in order to stabilize the magnetic field. A control stimulus was initially presented for 16 sec so that the first target stimulus epoch did not follow presentation of the uniform gray field. The alternating block was then replicated six times. Thus, in each run, a functional time series consisted of 104 gradient-echo echo planar imaging whole-brain scans (TR = 2000 msec, TE = 30 msec, FA = 90 deg, FOV = 192 × 192 mm ², 3 × 3-mm ² in-plane resolution, 36 interleaved axial slices of 3-mm slice thickness with no gaps). 

In Experiments 1B, 1C, 2B, and 2C, stimuli were presented randomly in event-related fashion ( Figures 1B and 1C). Each run started from a 30-sec presentation of the uniform gray field. No image was acquired for the first 14 sec. The uniform gray field was presented for 16 sec after the presentation of a motion stimulus for 16 sec in Experiments 1B and 2B, or a static stimulus for 8 sec in Experiments 1C and 2C. Each motion stimulus was presented four times in each run for Experiments 1B and 2B, and each static stimulus was presented at four different positions in each run for Experiment 1C and at four different orientations twice in a run for Experiment 2C. 

For purposes of registration, we also acquired high-resolution anatomical images for each subject (3-D magnetization-prepared rapid gradient-echo T1-weighted sequence, TR = 2000 msec, TE = 4.38 msec, 208 axial slices of 1-mm slice thickness with no gaps, 0.9375 × 0.9375 × 1-mm ³ voxels). 

Data analysis

For group analysis, image preprocessing and statistical analysis were performed on a personal computer (ThinkCentre A51p, IBM, NY) using MATLAB (The Mathworks, MA) and SPM2 (statistical parametric mapping software; Wellcome Trust Centre for Neuroimaging, London, UK). Image preprocessing included realignment, spatial normalization into a standard space (Montreal Neurological Institute (MNI) template) using affine and nonlinear transformations, and spatial smoothing with a Gaussian kernel (7-mm full width at half maximum). Global changes in blood oxygenation level-dependent (BOLD) signal were removed by scaling, with low-frequency drifts in the fMRI removed using a high-pass filter of 64 sec except for Experiments 1C and 2C, for which the high-pass filter of 48 sec was used. Condition effects were estimated by applying appropriate linear contrasts (Friston et al., 1995). Random-effects models were used to identify cortical areas with consistent activation across subjects. Loci of activation were defined as contiguous regions of voxels surpassing a minimum statistical threshold (P < 0.001 uncorrected). 

In addition, a paired t-test was conducted on each subject's contrast data obtained from two conditions in Experiment 1A, since the same control stimuli serving as baseline were used in the experiment. The statistical threshold was set at P < 0.001 uncorrected. 

Individual surface-based analysis and visualization was performed using in-house software written in VTK (Kitware, NY) and MATLAB (Ejima et al., 2003; Yamamoto et al., 2008). MRI and fMRI measurements and the procedures of cortical surface reconstruction and parcellation of the retinotopic areas of the visual cortex were performed prior to the present study. Then individual surface-based analyses were performed for nine subjects. First, data from fMRI scans were screened for technical artifacts such as head motion or baseline drift. All structural and functional volumes across multiple experimental sessions from a given subject were aligned to a common coordinate space for the subject's standard structural data by anatomical registration (Arun, Huang, & Blotstein, 1987) and head motion correction (Woods, Cherry, & Mazziotta, 1992). Next, the activated cortical regions were determined by performing a discrete Fourier transformation of the time course of change in activity in each voxel and an F-test of the ratio of stimulus-locked power to all other non-harmonic frequencies. Then, the phase of the signal at the stimulus frequency was used to identify regions activated by each SKE stimulus, taking into account the response delay on the order of 4–6 sec inherent in the hemodynamics. The response phase of the fMRI signal was mapped to the nearest node on the reconstructed cortical surface. 

Experiment 1: Peripherally and centrally eccentric-circle stimuli

Motion condition without blank intervals

Purpose

In order to reveal cortical areas involved in SKE, eccentric circles and concentric circles were used as a pair of SKE and control stimuli in each of the peripherally and centrally eccentric-circle (PEC and CEC) conditions, respectively ( Figure 2). Differences due to different physical arrangement between the SKE and control stimuli were balanced between the two conditions. First, activation by the SKE stimulus might result from a difference in impression of rotation speed between the SKE and control stimuli, since the SKE stimulus had more motion energy than the control stimulus in the PEC condition. Second, activation might occur due to a difference in positions to which subjects attended. For example, subjects might attend to the smallest circle of each stimulus. An experiment consisting of the PEC condition alone might thus not be able to identify cortical areas involved in SKE with precision. It was therefore necessary to add the CEC condition, in which the SKE stimulus appeared to rotate more slowly than the control stimulus, and in which stimulus arrangement was opposite to that in the PEC condition. 

Figure 2

 

Visual stimuli in Experiment 1. Three types of stimulus configurations were used. A red point in each panel indicates the fixation point. The PEC target stimulus consisted of five peripherally eccentric circles (A). On the other hand, the CEC target stimulus consisted of five centrally eccentric circles (B). These two stimuli had the same configuration. The control stimulus common to the PEC and CEC target stimuli consisted of five concentric circles (C). The centers of the smallest circles of the PEC and CEC target stimuli were located at eccentricities of 4.1 and 0.4 deg in visual angle, respectively. That of the control stimulus was located at an eccentricity of 1.8 deg. All the stimuli subtended a visual angle of 7.0 deg in radius. In Experiments 1A and 1B (motion conditions), the stimuli rotated in the fronto-parallel plane about the fixation point at a speed of 0.5 revolutions per sec. Rotations of the target stimuli evoked SKE and subjects perceived them as 3-D objects, while rotation of the control stimulus did not evoke SKE. In Experiment 1C (static condition), the static target and control stimuli were each presented at four different positions in a run. The directions from the fixation point toward the center of the smallest circle in each stimulus were 45, 135, 225, or 315 deg from the horizontal line.

Methods

Thirteen subjects (11 males and 2 females) participated in the experiment. 

Visual stimuli are shown in Figure 2. A pair of SKE and control stimuli was used for each condition. In the PEC condition, the SKE stimulus consisted of five peripherally eccentric circles resembling Benussi Circles (Musatti, 1924; Figure 2A). Subjects perceived a cone pointing outward or a funnel receding inward while viewing the SKE stimulus. On the other hand, in the CEC condition, the SKE stimulus consisted of five centrally eccentric circles (Figure 2B). Subjects perceived a cone pointing inward or a funnel receding outward while viewing the SKE stimulus. These two objects perceived did not differ in sign but in amplitude of depth variation, which was steeper in the CEC-SKE stimulus. The same control stimuli consisting of five concentric circles were used in both conditions (Figure 2C). Since the control stimuli did not evoke SKE at all, subjects perceived not 3-D objects but 2-D figures while viewing them. All the stimuli rotated about the fixation point at a speed of 0.5 revolutions per sec and subtended a visual angle of 7.0 deg in radius. 

Each subject underwent four runs for each condition. The experimental paradigm was a block design with alternating 16-sec presentations of SKE and control stimuli ( Figure 1A). Two-second transition periods, in each of which the stimulus of the former epoch was transformed smoothly into that of the latter by changing the eccentricities of the nested circles, were comprised of the final 1 sec of the former and first 1 sec of the latter epoch in order to suppress activation caused by sudden changes of stimuli. The fixation point turned green from red (1-sec duration) at 8 sec after the beginning of each epoch. Subjects were required to judge and to respond, by pressing buttons, whether a presented stimulus was perceived as a 2-D figure or 3-D object while the fixation point was green. Subjects who could not perceive the SKE stimulus as a 3-D object well enough in each condition or could not perform the task properly were required to participate in the experiment again. 

Results

Significant activation associated with the PEC-SKE stimulus was observed in the inferior temporal gyrus (ITG), middle temporal complex (hMT+), lateral occipital area (LO), V3A, ventral IPS (VIPS), and frontal eye field (FEF) in both hemispheres, as well as V3B and the dorsal IPS medial and anterior (DIPSM and DIPSA) in the left hemisphere, the fusiform gyrus (FG) and inferior precentral sulcus (IPreCS) in the right hemisphere, and the supplementary eye field (SEF), as shown in the upper panels of Figure 3A and Table 1. On the other hand, significant activation associated with the CEC-SKE stimulus was observed in FG, ITG, LO, and V3B in both hemispheres, as well as the left hMT+ and IPreCS, as shown in the lower panels of Figure 3A and Table 2. Moreover, significant deactivation associated with the CEC-SKE stimulus was observed in the right VIPS and left DIPSM. 

Figure 3

 

Results in Experiment 1A. (A) Activation patterns on group analysis mapped on a standard brain template showing left and right posterior views in the PEC (upper panels) and CEC (lower panels) conditions. Loci of activation are presented in Tables 1 and 2. The cold-color map indicates areas of activation for the SKE stimulus, while the warm-color map indicates areas of deactivation for the SKE stimulus in each condition. The threshold is set at P < 0.001 uncorrected. Color saturation indicates P value. (B) Activation patterns on individual analysis mapped on the inflated left hemispheres of three representative subjects in the PEC (upper panels) and CEC (middle panels) conditions. Representation of the visual field is denoted by colors in the lower panels. The red map indicates the most central visual field representation, while the purple map indicates the most peripheral. The cold-color map indicates areas of activation for the SKE stimulus, while the warm-color map indicates areas of deactivation for the SKE stimulus in each condition. The threshold was set at P < 0.001. Color saturation indicates P value. Names of early visual areas are also provided.

Table 1

 

Loci of activation determined by random-effects group analysis in the PEC condition of Experiment 1A. Regions indicated with asterisks are not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
FG				*	28	−58	−20	4.20
ITG	−46	−72	−12	5.68	44	−68	−14	4.41
hMT+	−46	−76	2	6.21	46	−76	0	5.86
LO	−40	−86	4	5.99	44	−84	2	5.30
V3B	−36	−86	8	5.77				*
V3A	−18	−94	12	6.90	20	−96	14	6.35
VIPS	−26	−90	30	6.61	22	−92	24	4.17
DIPSM	−28	−52	60	5.34				*
DIPSA	−44	−48	56	6.72				*
FEF	−36	−16	62	4.02	36	−14	56	5.28
SEF				*	6	10	48	5.13
IPreCS				*	48	6	28	4.46

Table 2

 

Loci of activation determined by random-effects group analysis in the CEC condition of Experiment 1A. T-values in VIPS and DIPSM indicate deactivation (control > target). Regions indicated with asterisks are not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
FG	−28	−46	−22	7.19	38	−46	−24	5.54
ITG	−36	−64	−18	11.11	38	−74	−16	4.29
hMT+	−42	−74	−6	9.31				*
LO	−44	−80	8	3.38	46	−82	8	4.67
V3B	−30	−88	6	9.78	40	−88	12	6.06
IPreCS	−46	−2	26	6.17				*
VIPS				*	10	−94	34	4.20
DIPSM	−22	−64	60	4.26				*

For early visual areas including V1, V2, V3, and ventral V4 (V4v), we performed individual surface-based analyses for nine subjects. Regions representing the peripheral visual field were significantly activated by the PEC-SKE stimulus and deactivated by the CEC-SKE stimulus in several subjects ( Figure 3B). In contrast, those representing the central visual field were significantly activated by the CEC-SKE stimulus and deactivated by the PEC-SKE stimulus. 

We further performed a paired t-test for each subject's contrast data obtained from the PEC and CEC conditions. Use of the same control stimuli, which served as baseline in each condition, enabled such a comparison. We defined the following three types of regions. The first were SKE-sensitive regions, in which the observed signal change was significant for both SKE stimuli. The second were regions that represent the peripheral visual field, in which the observed signal change was significantly higher for the PEC-SKE stimulus and lower for the CEC-SKE stimulus. The third were regions that represent the central visual field, in which the observed signal change was significantly higher for the CEC-SKE stimulus and lower for the PEC-SKE stimulus. It was expected that the last two types of regions would be affected by the stimulus arrangement. 

The results are shown in Figure 4A and Tables 3 and 4. The SKE-sensitive region included FG, ITG, hMT+, LO, V3B, and IPreCS in both hemispheres, and the left DIPSA. The regions representing the peripheral visual field included not only the peripheral visual field representations of early visual areas but also V3A and VIPS in both hemispheres, as well as the left DIPSM. The regions representing the central visual field included only parts of early visual areas. 

Figure 4

 

Summary of results in Experiment 1A and results in Experiment 1B. (A) Activation pattern determined by paired t-tests performed on results for the PEC and CEC conditions mapped on a standard brain template showing posterior view. Loci of activation are presented in Tables 3 and 4. Cortical regions significantly responding to both SKE stimuli are illustrated as SKE-sensitive (in magenta). Cortical regions responding to stimuli in which components were positioned more peripherally, that is, the SKE stimulus in the PEC condition and the control stimulus in the CEC condition, are illustrated as representing the peripheral visual field (in cyan). Cortical regions responding to stimuli in which components are positioned more foveally, that is, the control stimulus in the PEC condition and the SKE stimulus in the CEC condition, are illustrated as representing the central visual field (in purple). White characters indicate z-coordinates in MNI space. Color saturation indicates P value. (B) Percent BOLD signal changes of activated cortical areas in the left hemisphere shown in Tables 3 and 4. Those for the PEC- and CEC-SKE, and control stimuli are denoted by cyan, purple, and gray, respectively. Error bars indicate standard errors of the means.

Table 3

 

Loci of SKE-sensitive regions determined by paired t-test in Experiment 1A. A region indicated with an asterisk is not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
FG	−32	−62	−18	5.15	28	−58	−22	6.80
ITG	−44	−74	−10	9.27	44	−64	−18	8.26
hMT+	−44	−74	2	8.19	46	−76	0	4.34
LO	−40	−80	4	5.15	44	−82	4	6.19
V3B	−36	−86	8	8.94	40	−88	10	7.41
DIPSA	−46	−44	56	5.34				*
IPreCS	−42	−2	24	3.81	52	8	26	4.59

Table 4

 

Loci of regions representing the peripheral visual field determined by paired t-test in Experiment 1A. A region indicated with an asterisk is not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
V3A	−16	−94	10	8.74	20	−96	14	4.78
VIPS	−26	−90	30	4.03	22	−92	24	4.91
DIPSM	−22	−64	60	5.25				*

Motion condition with blank intervals

Purpose

The purpose of this experiment was twofold. First, the activation maps obtained in Experiment 1A ( Figure 3A) might have been due to a significant difference in negative BOLD signals between the SKE and control stimuli. Therefore, the same SKE and control stimuli as in Experiment 1A were presented in event-related fashion with blank intervals serving as baseline in order to confirm that the maps were in fact due to a significant difference in positive BOLD signals between them. Second, a difference in activation was observed in V3A, VIPS, DIPSM, and DIPSA between the PEC and CEC conditions in Experiment 1A. Subjects might have experienced more difficulty in viewing the CEC-SKE stimulus. To confirm this, subjects were required to judge and to respond not only whether a presented stimulus was perceived as a 2-D figure or 3-D object but also whether the impression of a 3-D object was strong or weak. 

Methods

Six subjects (4 males and 2 females) who had participated in Experiment 1A and seven new subjects (5 males and 2 females) participated in the experiment. Each subject underwent three runs. 

Stimuli were presented randomly in event-related fashion ( Figure 1B). The same SKE and control stimuli as in Experiment 1A were presented, with 16 sec per presentation ( Figure 2). The uniform gray field was presented for 16 sec after the presentation of a motion stimulus. Each stimulus was presented four times in each run. A functional time series consisted of 200 scans in each run. 

The fixation point turned green from red (1-sec duration) at 8 sec after the beginning of each epoch. Subjects were required to judge and to respond whether a presented stimulus was perceived as a 2-D figure or gave a strong or weak impression of a 3-D object while the fixation point was green. 

Results

Positive BOLD signals were observed in all the areas of activation observed in Experiment 1A ( Figure 4B). This finding indicates that these areas had higher positive BOLD signals for the SKE stimuli than for the control stimulus and that the activation maps obtained in Experiment 1A were not due to a significant difference in negative BOLD signals between the SKE and control stimuli. 

The results of the behavioral task are shown in Table 5. The impression of a 3-D object was stronger for the CEC-SKE stimulus than for the PEC-SKE stimulus. This indicates that the differences in activation pattern in V3A, VIPS, DIPSM, and DIPSA in Experiment 1A did not result from difficulty experienced in viewing the SKE stimuli. 

Table 5

 

Subjects' task performance in Experiments 1B and 1C. Means of percentages of “no”, “weak”, and “strong” responses in the judgment of the impression of a 3-D object for the target and control stimuli are shown.

Condition Impression	Motion			Static
No	Weak	Strong	No	Weak	Strong
PEC	12.2	19.2	68.6	43.3	49.0	7.7
CEC	5.1	11.5	83.3	39.4	50.0	10.6
Control	94.9	3.9	1.3	90.4	8.7	1.0

Static condition with blank intervals

Purpose

The activation maps in Experiment 1A ( Figure 3A) might have been due not to a significant difference in depth perception between the SKE and control stimuli but to a simple difference in 2-D image between them. We tested this possibility by presenting static target and control stimuli, none of which were expected to evoke SKE. Subjects were required to judge and to respond not only whether a presented stimulus was perceived as a 2-D figure or 3-D object but also whether the impression of a 3-D object was strong or weak in this experiment. 

Methods

The same thirteen subjects as in Experiment 1B participated in the experiment. Each subject underwent two runs. 

Stimuli were presented randomly in event-related fashion ( Figure 1C). The static versions of the SKE and control stimuli in Experiment 1A ( Figure 2) were presented for 8 sec per presentation. The uniform gray field was presented for 16 sec after the presentation of a static stimulus. Each stimulus was presented at four different positions in each run. The directions from the fixation point toward the center of the smallest circle in each stimulus were 45, 135, 225, or 315 deg from the horizontal line. A functional time series consisted of 152 scans in each run. 

The fixation point turned green from red (1-sec duration) at 4 sec after the beginning of each epoch. Subjects then performed the same task as Experiment 1B. 

We compared activities evoked by each static target stimulus and the control stimulus in the analysis. 

Results

Significant activation associated with the static PEC target stimulus was observed in the right FG ( Table 6). On the other hand, significant activation with the static CEC target stimulus was observed in the left ITG and right DIPSA ( Table 7). In fact, as shown in Table 5, subjects had some impression of a 3-D object for both static target stimuli. These results suggest that significant activation in ITG, FG, and DIPSA in Experiment 1A might have been due not only to the presence or absence of SKE but also to a simple difference in 2-D image between the SKE and control stimuli and a significant difference in depth perception between static versions of them. No significant activation associated with both target stimuli was observed in other cortical areas. 

Table 6

 

Locus of activation by the static PEC stimulus determined by random-effects group analysis in Experiment 1C. A region indicated with an asterisk is not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
FG				*	32	−50	−14	4.86

Table 7

 

Loci of activation by the static CEC stimulus determined by random-effects group analysis in Experiment 1C. Regions indicated with asterisks are not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
ITG	−42	−78	−18	5.45				*
DIPSA				*	48	−44	58	4.66

Discussion

Our findings suggest that significant activation in early visual areas is probably correlated with spatial properties of stimuli. In the PEC condition, lines of the SKE stimulus gathered more closely in the peripheral visual field than those of the control stimulus did. The opposite occurred in the CEC condition. A pattern of activation corresponding to this line gathering should be observed in these retinotopic areas. In other words, while the cortical regions associated with representation of the central visual field responded to the SKE stimulus and those associated with representation of the peripheral visual field responded to the control stimulus in the PEC condition, the responding areas were opposite to these in the CEC condition. It thus appears that early visual areas are not involved in SKE and that they exhibit precise retinotopic responses. This finding is consistent with that a previous study showing that early visual areas differ from hMT+ and LO in the roles they play in depth processing (Welchman, Deubelius, Conrad, Bülthoff, & Kourtzi, 2005). 

With use of paired t-tests, we could also dissociate widely extended cortical areas activated by the PEC-SKE stimulus into two regions. The first region included hMT+, LO, and V3B and was involved in SKE. The second was the intraparietal region including V3A, VIPS, and DIPSM and represented the peripheral visual field. However, the role of the intraparietal region in SKE remains unclear due to the difference in motion energy between the PEC- and CEC-SKE stimuli. It is possible that higher cortical activity associated with the PEC-SKE stimulus was observed simply because of its higher motion energy. The difference in motion energy is likely to be appropriate for identifying SKE-sensitive region but not necessarily regions representing the peripheral visual field. It was thus necessary to determine the role of the intraparietal region in SKE. Furthermore, we needed to identify cortical areas involved in SKE with greater precision using a static version of an SKE stimulus not giving the impression of a 3-D object, since subjects could have some impression of a 3-D object even for the static target stimuli. 

Crescent stimulus

Motion condition without blank intervals

Purpose

We needed to exploit a static version of the SKE stimulus from which subjects could not perceive a 3-D object, since they could perceive 3-D objects slightly even from the static target stimuli in Experiment 1C. Furthermore, we considered the following three questions. The first was whether activity in the intraparietal region, which was mapped as representing the peripheral visual field in Experiment 1A, resulted from a difference in motion energy or not. The second was whether a different SKE stimulus activated the tentative SKE-sensitive region in Experiment 1, i.e., hMT+, LO, and V3B, or not. Significant activation associated with the SKE stimulus should be observed in this region, if the region is actually SKE-sensitive. The third was whether cortical activity in early visual areas actually depended on stimulus arrangement or not. It was expected that use of a pair of stimuli that constantly subtend the same visual angle would enable canceling out of responses in early visual areas by subtraction. We therefore used a stimulus pair to minimize the difference between SKE and control stimuli ( Figures 5A and 5B). The pairs of stimuli had the same stimulus components, motion energy, and orbits of components. 

Figure 5

 

Visual stimuli in Experiment 2. Two types of stimulus configuration were used. A point in each panel indicates the fixation point. The target stimulus consisted of two different-sized crescent shapes (A). The control stimulus consisted of two same-sized crescent shapes (B) and was created by rearranging the target stimulus. The red, green, and blue pieces of the control stimulus are rotated counterclockwise 33, 213, and 180 deg about the fixation point in comparison with those of the target stimulus, respectively (C). Both stimuli subtended a visual angle of 7.0 deg in radius. In Experiments 2A and 2B (motion conditions), the stimuli rotated in the fronto-parallel plane about the fixation point at a speed of 1.0 revolution per sec. The rotation of the target stimulus evoked SKE and subjects perceived it as a 3-D object, while the rotation of the control stimulus did not evoke SKE and subjects perceived it simply as a 2-D figure. In Experiment 2C (static condition), the static target and control stimuli were each presented at four different orientations twice in a run. The orientations of each stimulus were 0, 90, 180, or 270 deg.

Methods

The same thirteen subjects (11 males and 2 females) as in Experiment 1A participated in the experiment. 

The target stimulus consisted of two crescent shapes (Sumi, 1989) while the control stimulus was comprised of two differently arranged crescent shapes consisting of the same four components organizing the target stimulus (Figure 5C). Subjects perceived a cylinder pointing outward and protruding out of the screen toward them while viewing the SKE stimulus. The rotating control stimulus did not evoke SKE, similar to Experiment 1A. All the stimuli rotated about the fixation point at a speed of 1.0 revolution per sec and subtended a visual angle of 7.0 deg in radius. 

Each subject underwent four runs. Although the experimental paradigm was the same as in Experiment 1A, the stimulus of the former epoch faded out for the last 1 sec and that of the latter faded in for the first 1 sec in each 2-sec transition period. Subjects performed the same task as in Experiment 1A. Subjects who could not perceive the SKE stimulus as a 3-D object well enough in each condition or could not perform the task properly were required to participate in the experiment again. 

Results

Significant activation associated with SKE was observed in ITG, hMT+, V3A, V3B, VIPS, the parieto-occipital IPS (POIPS), DIPSM, DIPSA, FEF, SEF, and IPreCS in both hemispheres, the left FG and LO as shown in Figure 6A and Table 8. No significant activation associated with SKE was observed in early visual areas ( Figure 6B). 

Figure 6

 

Results of Experiments 2A and 2B. (A) Activation pattern on group analysis mapped on a standard brain template showing left and right posterior views. More details on loci of activation are presented in Table 8. The cold-color map indicates areas of activation for the SKE stimulus, while the warm-color map indicates areas of deactivation for the SKE stimulus. The threshold is set at P < 0.001 uncorrected. Color luminance indicates P value. (B) Activation pattern on individual analysis mapped on the inflated left hemispheres of the same subjects as in Figure 3B. Representation of the visual field is denoted by colors in the bottom panel. The red map indicates the most central visual field representation, while the purple map indicates the most peripheral. The cold-color map indicates areas of activation for the SKE stimulus, while the warm-color map indicates areas of activation for the control stimulus. The threshold was set at P < 0.001. Color saturation indicates P value. Names of early visual areas are also provided. (C) Percent BOLD signal changes of activated cortical areas in the left hemisphere shown in Table 8. Those for the SKE and control stimuli are denoted by cyan and orange, respectively. Error bars indicate standard errors of the mean.

Table 8

 

Loci of activation determined by random-effects group analysis in Experiment 2A. Regions indicated with asterisks are not above threshold ( P < 0.001 uncorrected).

	Left				Right
x	y	z	T	x	y	z	T
FG	−28	−52	−14	4.64				*
ITG	−36	−74	−10	5.48	48	−72	−8	4.33
hMT+	−44	−74	4	4.86	48	−72	2	8.57
LO	−40	−80	4	4.37				*
V3B	−36	−86	6	7.30	34	−86	4	4.46
V3A	−24	−94	18	6.28	26	−88	12	4.86
VIPS	−22	−92	32	4.86	22	−92	26	5.86
POIPS	−22	−74	40	4.36	26	−74	44	5.23
DIPSM	−24	−64	50	6.18	34	−58	54	4.89
DIPSA	−34	−46	50	4.41	42	−40	50	5.47
FEF	−34	−12	64	4.68	24	−2	58	4.68
SEF	0	−2	58	4.19	6	−4	64	4.56
IPreCS	−48	0	34	4.22	52	6	32	4.35

Motion condition with blank intervals

Purpose

A control experiment including blank intervals was also performed for the crescent SKE stimulus for the same purpose as in Experiment 1B. We confirmed that the activation map obtained in Experiment 2A was due to a significant difference in positive BOLD signals between the SKE and control stimuli. 

Methods

The same thirteen subjects as in Experiment 1B participated in the experiment. Each subject underwent three runs. 

Stimuli were presented randomly in event-related fashion ( Figure 1B). The same SKE and control stimuli as in Experiment 2A were presented, with 16 sec per presentation ( Figures 5A and 5B). Each stimulus was presented four times in each run. A functional time series consisted of 136 scans in each run. 

The fixation point turned in the same way, and subjects performed the same task as in Experiment 1B. 

Results

Positive BOLD signals were observed in all the areas of activation observed in Experiment 2A ( Figure 6C), indicating that the activation map obtained in Experiment 2A was not due to a significant difference in negative BOLD signals between the SKE and control stimuli. 

The results of the behavioral task are shown in Table 9. Subjects had the strong impression of a 3-D object with the SKE stimulus. 

Table 9

 

Subjects' task performance in Experiments 2B and 2C. Means of percentages of “no”, “weak”, and “strong” responses in the judgment of the impression of a 3-D object for the target and control stimuli are shown.

Condition Impression	Motion			Static
No	Weak	Strong	No	Weak	Strong
Target	10.3	13.5	76.3	86.5	12.5	1.0
Control	82.1	18.0	0.0	94.2	3.9	1.9

Static condition with blank intervals

Purpose

A control experiment using static versions of the SKE and control stimuli in Experiment 2A was also performed for the crescent SKE stimulus for the same purpose as in Experiment 1C. We tested the possibility that the activation map shown in Experiment 2A was due not to a significant difference in depth perception between the SKE and control stimuli but to a simple difference in 2-D image between them. 

Methods

The same thirteen subjects as in Experiment 2B participated in the experiment. Each subject underwent one run. 

Stimuli were presented randomly in event-related fashion ( Figure 1C). The static versions of SKE and control stimuli in Experiment 2A were presented for 8 sec per presentation ( Figures 5A and 5B). Each stimulus was presented at four different orientations twice in a run. The orientations of each stimulus were 0, 90, 180, or 270 deg. A functional time series consisted of 200 scans. 

The fixation point turned in the same way, and subjects performed the same task as in Experiment 2B. 

We compared activities evoked by the static target control stimuli in the analysis. 

Results

No significant activation associated with the static target stimulus was observed in any area of activation observed in Experiment 2A, indicating that the activation map obtained in Experiment 2A was due to a significant difference in depth perception between the SKE and control stimuli and not to a simple difference in 2-D image between them. 

The results of the behavioral task are shown in Table 9. Subjects for the most part perceived the static target stimulus as a 2-D figure. 

Discussion

Cortical activities in early visual areas were exactly canceled out by the SKE and control stimuli, suggesting that activation of these areas is affected not by SKE but by areas stimulated in the visual field. As regards higher visual areas, although the physical differences between the SKE and control stimuli were minimized to the extent possible, SKE activated numerous cortical areas, including both the region including hMT+, LO, and V3B, and intraparietal region. It thus appears that, while the region including hMT+, LO, and V3B is certainly SKE-sensitive, the intraparietal region neither represents the peripheral visual field nor is simply motion-sensitive. 

Here the question arises why the PEC and crescent SKE stimuli, but not the CEC-SKE stimulus, commonly activated the intraparietal region including V3A, VIPS, and DIPSM. Although previous studies have reported that the intraparietal region is motion-sensitive (Sunaert, Van Hecke, Marchal, & Orban, 1999) and 3-D SFM-sensitive (Kriegeskorte et al., 2003; Orban et al., 1999; Paradis et al., 2000; Vanduffel et al., 2002), our findings for the CEC condition are inconsistent with these conclusions. If activation had differed only between the PEC and CEC conditions, this might have suggested that subjects had attended to the smallest circle of the SKE stimulus in each condition. However, the crescent SKE stimulus, in which no difference existed in stimulus arrangement between the SKE and control stimuli, activated the intraparietal region. 

General discussion

Controlling motion and contrast energies as well as stimulus presentation paradigm, we attempted to identify the cortical areas involved in SKE. We observed different patterns of cortical activity among experimental conditions. This suggests that widely extended cortical areas activated by SKE stimuli are separable into several functionally distinct regions. Early visual areas (V1, V2, V3, and V4v) exhibited patterns of activation that depended on stimulus arrangement, suggesting that they are not directly involved in SKE. The region including hMT+, LO, and V3B was activated by all SKE stimuli used in the present study, suggesting that it plays a central role in SKE. The intraparietal region (V3A, VIPS, and DIPSM) was not activated by all the SKE stimuli but by the PEC and crescent SKE stimuli alone. On the other hand, ITG, FG, and DIPSA were activated by all the SKE stimuli, but they were also activated by the static version of the PEC- or CEC-SKE stimulus. These results indicate that the intraparietal region, ITG, FG, and DIPSA are not simply SKE-sensitive. We discuss below the roles of areas including hMT+, LO, and V3B and the activity in the intraparietal region, ITG, FG, and DIPSA. 

hMT+, LO, and V3B as higher order areas for 3-D object reconstruction

Our results have suggested that hMT+, LO, and V3B play a central role in depth processing in SKE. Our findings are consistent with recent studies showing that cortical areas sensitive to 3-D SFM include the lateral occipital sulcus and hMT+ (Kriegeskorte et al., 2003; Orban et al., 1999; Paradis et al., 2000; Peuskens et al., 2004; Vanduffel et al., 2002). As noted above, however, SKE differs from general 3-D SFM in the absence and presence of velocity gradients. In fact, these previous studies investigated 3-D SFM based on the assumption that velocity gradients are necessary for 3-D SFM. In recent single-cell studies, moreover, MT neurons of monkeys were found to be selective for velocity gradients (Nguyenkim & DeAngelis, 2004; Xiao, Marcar, Raiguel, & Orban, 1997). In contrast, the present study revealed 3-D object reconstruction from motion using 3-D SFM lacking velocity gradients, i.e., SKE. These findings suggest that 3-D SFM is not necessarily linked to motion in 3-D space. It thus appears that general motion-dependent 3-D object processing is achieved in hMT+, LO, and V3B regardless of velocity gradients. This appears consistent with the findings of a recent study by Hasson, Harel, Levy, and Malach (2003). They noted that dorsal and ventral occipito-temporal (DOT and VOT) cortex is comprised of higher order object perception areas and that DOT is motion-sensitive. A portion of DOT appears to correspond to hMT+, LO, and V3B as identified in the present study. 

The higher order functioning of hMT+, LO, and V3B in 3-D object reconstruction processes may not be limited to motion cues and may extend to all depth cues. Previous studies have suggested the possibility that the neural mechanism of KDE is common to that for binocular disparity (Cornilleau-Pérès & Droulez, 1993; Nawrot & Blake, 1989, 1991, 1993; Tittle & Braunstein, 1993). In addition, recent fMRI studies have shown that hMT+ and LO are involved in 3-D object perception from various single depth cues (Kourtzi, Bülthoff, Erb, & Grodd, 2002; Kourtzi, Erb, Grodd, & Bülthoff, 2003; Moore & Engel, 2001) and multiple combined depth cues (Chandrasekaran, Canon, Dahmen, Kourtzi, & Welchman, 2007; Welchman et al., 2005). 

Activity in the intraparietal region

Many studies have found a relationship between the intraparietal region and depth processing. It has been reported that the human intraparietal region is 3-D SFM-sensitive (Kriegeskorte et al., 2003; Orban et al., 1999; Paradis et al., 2000; Vanduffel et al., 2002). Moreover, the monkey caudal intraparietal (CIP) area represents 3-D surface orientation from various depth cues such as binocular disparity, texture gradients, and shading (Shikata et al., 2001; Taira, Nose, Inoue, & Tsutsui, 2001; Taira, Tsutsui, Jiang, Yara, & Sakata, 2000; Tsutsui, Jiang, Yara, Sakata, & Taira, 2001; Tsutsui, Sakata, Naganuma, & Taira, 2002). On the other hand, other studies have reported that the intraparietal region is also activated by spatial attention, eye movement, pointing movement, and mental rotation (Astafiev et al., 2003; Corbetta et al., 1998; Grosbras, Laird, & Paus, 2005; Luks & Simpson, 2004; Podzebenko, Egan, & Watson, 2002). These findings indicate that parietal cortex, including the IPS, is a typical higher order, multimodal region of cortex involved in a surprisingly large series of cognitive functions (Orban et al., 2006). 

In the present study, the intraparietal region was activated when subjects viewed the PEC and crescent SKE stimuli and was deactivated when they viewed the CEC-SKE stimulus. The difference in activation pattern for the SKE stimuli may be ascribed to a difference in the net effect of several factors in the intraparietal region. 

First, it might be ascribed to a difference in eye movement. Gnadt and Mays (1995) have shown that neurons in the monkey lateral intraparietal (LIP) area are more directly dependent on eye-movement parameters in 3-D space. Meanwhile, Ringach, Hawken, and Shapley (1996) have found that perception of 3-D SFM causes binocular eye movements similar to those that occur when a stimulus really rotating in 3-D space is viewed. In order to assess the effect of eye movement, we performed an experiment on eye movement in the PEC, CEC, and crescent conditions for three subjects who had participated in all fMRI experiments. However, no different eye movement patterns were observed either between the SKE and control epochs in any conditions or among the conditions, as shown in Figure 7. It is thus suggested that the differential activation in the intraparietal region does not result from direct eye movement. Yet we have to pay attention to the possibility that eye movement-related activity might be observed in the monkey LIP, which is thought to correspond to the human DIPSM (Orban et al., 2006), and human DIPSM to suppress eye movement even if subjects do not actually move their eyes. 

Figure 7

 

Eye movement patterns measured for three subjects in the PEC (A), CEC (B), and crescent (C) conditions. Epochs of the SKE stimulus presentation are indicated in cyan, and epochs of the control stimulus presentation are indicated in magenta. The eye movement patterns did not differ either between the epochs in any conditions or among the conditions.

Second, it might be ascribed to a difference in features of stimuli other than depth structure from the stimuli. The intraparietal region is simply motion-sensitive regardless whether it is relevant to depth processing or not (Sunaert et al., 1999). In the present study, there are differences in motion speed, apparent size, eccentricity, and curvature of surface between the PEC- and CEC-SKE stimuli. These differences might be ascribed to the difference in activation pattern of the intraparietal region for the two stimuli. 

Third, it might be ascribed to differences in the calculation of object motion along 3-D trajectory and attentional load in tracking 3-D object motion. Peuskens et al. (2004) have indicated that DIPSA is sensitive to both 3-D shape and 3-D motion while DIPSM is sensitive to 3-D shape, and Jovicich et al. (2001) have demonstrated a relationship between the anterior IPS and attentional load. On the other hand, Sakata et al. (1998) have found neurons showing a strong response to elongated stimuli and tuning the orientation of the longitudinal axis in the monkey CIP. A difference in 3-D trajectory of longitudinal axis among 3-D objects induced by SKE stimuli might have resulted in the different activations among the experimental conditions in the intraparietal region. 

Activity in ITG, FG, and DIPSA

ITG, FG, and DIPSA were activated by the static version of the PEC- or CEC-SKE stimulus, which could give the weak impression of a 3-D object, but not by that of the crescent SKE stimulus. Tables 5 and 9 show that the percentages of “weak” or “strong” 3-D object impression for the static versions of the SKE stimuli are 56.7% in the PEC, 60.6% in the CEC, and 13.5% in the crescent condition, respectively. The difference in activation for the static versions may be ascribed to the difference in the 3-D object impressions for these stimuli. It is suggested that ITG, FG, and DIPSA may be responsive to 3-D perception regardless of static or dynamic stimuli. 

We actually observed activation associated with 3-D perception in ITG and FG even though we compared the perception of a 3-D object with that of a 2-D figure in each dynamic condition. This suggests that these areas may process not only visual stimuli involving such factors as face perception (Kriegeskorte et al., 2003) and attention to surface texture (Peuskens et al., 2004) but also general 3-D volume information. 

Murray et al. (2003) have pointed out that the parietal shape area (PSA) showed increased signal to 3-D relative to 2-D figures. The PSA appeared to be DIPSA in the present study, considering similarity in anatomical position and its function. In addition, Durand et al. (2007) mentioned that the monkey anterior intraparietal (AIP) area and the anterior LIP are more specifically engaged in extracting the 3-D shapes of objects. The human DIPSA, which is thought to be homolog of the monkey AIP, would be also engaged in extracting the 3-D shapes of objects for control of grasping and so forth. 

Acknowledgments

This study was supported by grants from JSPS Research Fellowships for Young Scientists (17-2093), the 21st Century COE Program, D-2 to Kyoto University, MEXT, Japan (H. Y. and Y. E.), and Grand-in-Aid for Scientific Research from JSPS (20530663). We thank two reviewers, Hiroki Yamamoto and Hiroshi Ban, for their helpful suggestions and discussion, and Nobukatsu Sawamoto, Yoshio Ohtani, and Atsuya Shimba for their assistance in performing experiments. 

Commercial relationships: none. 

Corresponding author: Tetsuya Yamamoto. 

Email: [e-mail]. 

Address: Yoshida-Nihonmatsu-cho, Sakyo-ku, Kyoto 606-8501, Japan.