Spatiotemporal feature attribution for the perception of visual size

Abstract

This study examined the role of spatiotemporal feature attribution in the perception of the visual size of objects. A small or a large leading disk, a test disk of variable size, and a probe disk of a fixed size were sequentially presented at the same position for durations of 16.7 ms with interstimulus intervals of 117 ms. Observers compared the visual size of the test with the probe disk. The size of the test disk was underestimated and overestimated when the test followed small and large leading disks, respectively ( Experiment 1). These modulations of visual size occurred even when disks were sequentially presented so as to invoke apparent motion ( Experiment 2). Furthermore, when two streams of apparent motion consisting of the three types of disk were diagonally overlapped, modulation of visual size occurred in accordance with the size of the attended leading disk ( Experiment 3). Retinotopic and non-retinotopic feature attribution and the related attentional mechanisms are discussed.

Introduction

Integrating visual features is a fundamental task for the visual system. In the brain, visual features such as color, motion, and orientation are separately analyzed; later, they are integrated so as to form an object representation with multiple features. However, it has been found that the visual system occasionally fails to integrate visual features in a spatiotemporally appropriate way. For example, inappropriate binding of visual features is reported in illusory conjunction (Treisman & Schmidt, 1982) and was proposed to occur in crowding (for example, see Baldassi, & Burr, 2000; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Pelli, Palomares, & Majaj, 2004; Põder, 2006; Wilkinson, Wilson, & Ellemberg, 1997). 

A recent study has shed light on the spatiotemporal integration of visual features. Using a Ternus–Pikler display, Ogmen, Otto, and Herzog (2006) showed that a vernier offset of a line in the first frame was attributed to another line in the second frame. The Ternus–Pikler display has been phenomenally investigated by Gestalt Psychologists (Ternus, 1926; Pikler, 1917; for a recent review, see Petersik & Rice, 2006). In this display, three elements horizontally aligned in the first frame move toward the right or the left by an interelement distance in the second frame. Either of two motion percepts is obtained, depending on the interstimulus interval (ISI): Only outside elements are perceived as having moved when the ISI is short (for example, 0 ms), whereas both inner and outer elements are perceived as having been translated when the ISI is long (for example, 100 ms). 

Interestingly, Ogmen et al. (2006) showed that the pattern of feature attribution was dependent on the ISI between frames. Specifically, when the ISI was 0 ms, the vernier offset in the inner line in the first frame was attributed to the line in the second frame at retinotopically the same position. However, when the ISI was 100 ms, the vernier offset was attributed to the line that was presented at a different spatial position from the line with the vernier offset in the first frame but which perceptually corresponded in apparent motion. Their results indicate that visual features can be non-retinotopically remapped in accordance with motion correspondence. That is, visual features may be attributed across space and time. These results have been supported by a number of studies (Otto, Ogmen, & Herzog, 2006; Scharnowski, Hermens, Kammer, Ogmen, & Herzog, 2007). 

Such feature attribution can also be observed in simpler displays. For example, Wilson and Johnson (1985) showed that when a line target with a gap was masked by a line mask without a gap, the gap was perceived in the line mask (a transposition effect). Moreover, Herzog and Koch (2001) demonstrated that a masked feature, such as orientation, vernier offset, and motion of gratings, was seen in a mask stimulus of a grating. These studies indicate that feature attribution is possibly not limited to the vernier offset, as shown by Otto et al. (2006) but that it can be extended to several visual features. 

The retinotopic and non-retinotopic attribution effect can be explained by slow and fast buffers of the visual system (Herzog, Scharnowski, & Hermens, 2007; Scharnowski et al., 2007). Retinotopic feature attribution occurs when the first and the second features are successively presented with a short ISI that is sufficient for an integration neuron to merge both features: The integration neuron thus functions as a fast buffer in a retinotopic manner. For non-retinotopic feature attribution, on the other hand, Scharnowsiki et al. (2007) assumes a slow, object-based buffer, in which the information in the first frame is stored. The stored information is merged with the information in the second frame in an object-based manner: This leads to non-retinotopic feature attribution across space and time. 

The present study examined whether feature attribution across space and time occurs during the perception of visual size. Visual size is a cue to approaching/receding motion (Beverley & Regan, 1979; Regan & Beverley, 1978; Regan & Hamstra, 1993; Shirai & Yamaguchi, 2004). However, regardless of the importance of visual size in a dynamic context, it remained to be clarified how visual size is perceived and integrated into a coherent object. Previous studies have shown retinotopic and non-retinotopic feature attribution effects in the perception of a vernier offset (Ogmen et al., 2006), color (Nishida, Watanabe, Kuriki, & Tokimoto, 2007), motion (Herzog & Koch, 2001), orientation (Herzog & Koch, 2001), shape (Treisman & Schmidt, 1982), and line gap (Wilson & Johnston, 1985). Here, I supposed that clarifying the integration of visual size in a dynamic situation causing feature attribution effects is likely to be essential for understanding how visual features in moving objects are integrated across space and time since a dynamic change in visual size often leads to percepts of approaching/receding motion. By examining the effect of the size of a leading disk on that of a subsequent test disk, I show that retinotopic feature attribution occurs in the perception of the visual size of a disk (Experiment 1). I also determined whether feature attribution in the perception of visual size was non-retinotopic (Experiment 2). Moreover, I show that non-retinotopic feature attribution occurs in accordance with the size of the attended leading disk. 

Experiment 1

Experiment 1 tested whether the size of a leading disk could affect the perceived size of a test disk that immediately trailed the leading disk. Using a method of constant stimuli and the estimation of PSE (points of subjective equality) I investigated the quantitative aspects of feature attribution for perceiving visual size. I tried to clarify whether the size of the test disk was identical to that of the leading disk, or if the sizes of both disks were different. 

Methods

Observers

Five observers including the author (TK) participated in this experiment. Apart from the author, the observers were unaware of the purpose of experiments. All had normal or corrected-to-normal eyesight. 

Apparatus

Stimuli were presented on a 19-in. CRT monitor (RDF193H, Mitsubishi, Japan) with a resolution of 1024 × 768 pixels nd a refresh rate of 60 Hz. A Macintosh computer (MacBook, Apple) connected to the CRT monitor controlled stimulus presentation as well as data collection. A chin and head rest (TKK930A, Takei, Japan) was used to stabilize observers' visual fields. 

Stimuli

Stimuli were generated and presented using MATLAB and Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). A stimulus movie had a fixation cross [two orthogonal black (1 cd/m²) lines with 0.1 deg width × 1 deg length], leading, test, and probe green disks [CIE xyY (0.289, 0.587, 21.8)], and a gray background (5.2 cd/m²). Each trial started with a fixation cross presented for 500 ms. A sequence comprising the leading test and the probe disc then followed. Each disc was presented for 16.7 ms. The interstimulus interval between two subsequent discs was 117 ms. Figure 1 shows a schematic diagram of the stimulus presentation, and Movie 1 includes demonstrations of stimulus sequences. The fixation cross was presented at the center of the display, and the centers of leading, test, and probe disks were positioned 5.7 deg above or below the center of the fixation cross. The radius of the leading disk was 0.94 or 2.84 deg. The radius of the test disk was randomly selected from seven alternatives (1.18, 1.42, 1.66, 1.9, 2.14, 2.38, or 2.62 deg). The radius of the probe was always 1.9 deg. Thus, the differences between test and probe disks were −0.72, −0.48, −0.24, 0, 0.24, 0.48, and 0.72 deg. 

Figure 1

 

Schematic diagrams of the stimulus presentation in each experiment.

 

Movie 1

 

A demonstration for small and large leading disk conditions in Experiment 1.

Procedure

Observers initiated each trial by pressing the spacebar on a keyboard externally connected to a MacBook. After the keypress, a stimulus movie was presented. During the presentation of the movie, observers maintained their gaze on the fixation cross. After the observation, they reported whether the probe disk was smaller or larger than the test disk using assigned keys. They were urged to respond accurately. Each observer received 280 trials consisting of 2 leading disks with different radii × 7 test disks with different radii; there were 20 replications. The order of trials was randomized across observers. 

Results

A psychometric function was fitted to the data for each observer to estimate the point of subjective equality (PSE) for visual size between test and probe disks. Figures 2A and 2B show the averaged proportion of trials in which the probe was judged to be larger than the test and the individual and averaged PSEs across observers, respectively. A two-tailed t test showed that there was a significant difference in PSEs between conditions with small and large leading disks ( t ₄ = 5.97, p < .005). 

Figure 2

 

Results of Experiment 1. (a) Proportions of trials in which the probe was perceived to be larger than the test disk. Error bars denote standard errors of the means. (b) Individual and averaged PSEs for the test disk. Error bars denote 95% confidence intervals.

Discussion

The results showed that the size of a leading disk strongly modulated the perception of a trailing disk's size. This indicates that retinotopic feature attribution occurs during the perception of visual size. 

Experiment 2

Does a non-retinotopic feature attribution also occur during the perception of visual size? To answer this question, I used stimuli with a translational apparent motion: The leading disk was presented at a spatiotemporally different position from the test disk in order to generate a horizontal apparent motion, while the ISI was kept constant (117 ms). Using these stimuli, I examined whether non-retinotopic feature attribution occurred in the perception of visual size. 

Method

Observers

Four observers including the author (TK) participated in this experiment. Apart from the author, the observers were unaware of the purpose of experiments. All had normal or corrected-to-normal eyesight. 

Apparatus

The apparatus was identical to the one used in Experiment 1. 

Stimuli

The stimulus movie was identical to that used in Experiment 1 except for the following: leading, test, and probe disks were presented in a translational apparent motion. Initially, a leading disk was presented 2.83 deg to the left or right of and 5.6 deg above or below the fixation cross. Second, the test disk was presented 5.6 deg above or below the fixation cross. Finally, the probe was presented 2.83 deg to the right or left of and 5.6 deg above or below the fixation cross (please see Figure 1 and Movie 2). Thus, the direction of apparent motion (leftward or rightward) as well as the vertical position (above or below of the fixation cross) was randomly determined from trial to trial. 

 

Movie 2

 

A demonstration for small and large leading disk conditions in Experiment 2.

Procedure

The procedure was also identical to that of Experiment 1 except for the following: Observers were asked to judge whether the probe disk was larger or smaller than the test disk, even though they were presented in different positions from one another. They received 280 trials, including 2 radii of leading disks × 7 radii of test disks × 20 replications. 

Results

As in Experiment 1, I estimated the PSEs for visual size between the test and the probe disks. Figures 3A and 3B show the averaged proportion of trials in which the probe was judged to be larger than the test and the individual and averaged PSEs across observers, respectively. A two-tailed t test showed that there was a significant difference in PSE between conditions with large and small leading disks ( t ₃ = 3.28, p < .05). 

Figure 3

 

Results of Experiment 2. (a) Proportions of trials in which the probe was perceived to be larger than the test disks. Error bars denote standard errors of the means. (b) Individual and averaged PSEs for the test disk. Error bars denote 95% confidence intervals.

Discussion

The results showed a robust non-retinotopic feature attribution: The size of a leading disk altered the perceived size of a test disk, even when they were presented at different places. Thus, the results indicate that non-retinotopic feature attribution is not confined to the vernier offset but is extendable to visual size. 

In this experiment, a small leading disk caused a greater magnitude of feature attribution than a large leading disk. In Experiment 1, the magnitudes of feature attribution in both small and large leading disk conditions were significantly different from a null effect, as supported by the analysis with 95% confidence intervals (see Figure 2B). However, in Experiment 2, feature attribution with a large leading disk was not statistically different from null (see Figure 4B). It remains unclear why this difference in the magnitude of feature attribution occurred across experiments. I speculate that there are two possible sources of this difference. First, in the large leading disk condition, the separation between disks was smaller than that in the small leading disk condition. Thus, the judgment of size in the former condition might be relatively easy and accurate. Second, the area ratio between the leading and the test disks might contribute to the difference in the magnitude of feature attribution: In Experiment 1, the leading and test disks were presented at a similar position; thus, the absolute difference in radius between the leading and the test disks appeared to be useful for the calculation of size changes; the difference (0.95 deg in radius) was identical between the large and the small leading conditions, probably resulting in an equal amount of feature attribution. However, in Experiment 2, it is likely that absolute information was difficult to use because the disks were presented at distant places. Rather, the area ratio between leading and test disks was possibly the source of information. Thus, I suggest that the large area ratio (1:4 between leading and test disks) in the small leading disk condition might cause a greater magnitude of feature attribution compared with the small area ratio (1:2.25) in the large leading disk condition. 

Figure 4

 

The results of the additional experiment (gray bars) compared with those of Experiment 2 (white bar). Error bars denote 95% confidence intervals.

Additional experiment

Whether non-retinotopic feature attribution is consistently observed across ISIs between disks is an interesting issue since investigations with ISIs that are too long or too short, which do not support apparent motion, are useful to clarify whether apparent motion is a necessary condition for feature attribution in the perception of visual size to occur. To clarify this issue, in an additional experiment, I employed shorter (11.8 ms) and longer (424.8 ms) SOAs, neither of which support apparent motion (Strybel & Vatakis, 2004) besides the 117-ms ISI used in Experiment 2. I used only the smaller leading disk (0.94 deg) since, in Experiment 2, I observed a significant effect of the leading disk only in the small size condition. The duration of the frame containing this disk was 11.8 ms. Other aspects of the apparatus, stimuli, and procedure were identical to those of Experiment 2. Four people who did not participate in Experiment 2 were recruited as observers. 

The results are shown in Figure 4. Data from each observer were fitted using psychometric functions to calculate the PSE of the test disk for the probe disk. As illustrated in the error bars of Figure 4, I observed significant modulation of visual size in the longer ISI condition ( p < .05) but not in the shorter ISI condition ( p > .05), although there was no significant difference in PSE between the shorter and the longer ISI conditions ( t(4) = 0.49, p > .5). 

It is worth noting that the magnitude of size modulation in this additional experiment was considerably reduced compared with that in Experiment 2. It was difficult to directly assess the differences between the results of Experiment 2 and the additional experiment by ANOVA. However, I conducted a two-sample t test for each pair of ISI conditions following confirmation that the variances across conditions were not different from each other using an F-test ( Fs(3, 4) = 2.17 and 3.27, for pairs of ISIs of 11.8 and 117 ms, and 117 and 425 ms, respectively, p > .05). There was a significant difference in PSEs between 11.8- and 117-ms ISIs ( t(7) = 2.55, p < .04). On the other hand, there was only a marginally significant difference in PSEs between 117- and 425-ms ISIs ( t(7) = 2.29, p < .06). Thus, the results showed that the magnitude of feature attribution was greatest in the 117-ms ISI condition; this indicates that feature attribution in the perception of visual size has an optimal ISI that is also optimal for apparent motion (Strybel & Vatakis, 2004). However, since there was only a marginally significant difference in PSEs between 117- and 425-ms ISIs, this effect seems to occur up to ISIs of at least 425 ms. 

Experiment 3

This experiment additionally examined whether feature attribution for visual size was sensitive to attentional modulation. Otto et al. (2006) showed that the feature attribution of a vernier offset occurred only in the attended path. Their results imply the involvement of attention in feature attribution. Here, I asked whether feature attribution of the size of disks occurred only in the attended path. I presented a stimulus movie with two orthogonal motion paths, in which large and small leading disks were concurrently presented in the first frame, a test disk was completely overlapped at the center of the display, and two probe disks were presented along each motion path. Observers were asked to attend to one of the motion paths and to judge whether the test disk was larger or smaller than the probe disk. No modulation of visual size would be observed if an attention-free integration process underlay the feature attribution of a disk's size; on the other hand, a modulatory effect on visual size would be observed if feature attribution for the perception of visual size was based on attentive tracking. 

Method

Observers

Five observers, including the author (TK), participated in this experiment. Apart from the author, the observers were unaware of the purpose of experiments. All had normal or corrected-to-normal eyesight. 

Apparatus

The apparatus was identical to that of Experiment 1. 

Stimuli

The stimuli were identical to those in Experiment 2 except for the following: Initially, two leading disks with large or small radii were presented 2.8 deg left and right of and 5.6 deg above the fixation cross. When the size of the leading disk at one side was 0.95 deg, the size of the disk at the other side was 2.84 deg. Second, a test disk with a radius of 1.43, 1.9, or 2.37 deg was presented behind the fixation cross. Third, probe disks with radii of 1.9 deg were presented 2.8 deg left and right of and 5.6 deg below the fixation cross (please see Figure 1 and Movie 3). Thus, a stimulus movie had two streams of apparent motion, in right-diagonal and left-diagonal directions. 

 

Movie 3

 

A demonstration for stimuli in Experiment 3.

Procedure

The procedure was identical to that of Experiment 2 except for the following: Observers received two sessions. Two observers were asked to attend to the right-diagonal apparent motion at the first session and to the left-diagonal apparent motion at the second session; three observers were asked to attend in the reverse manner. Observers judged whether the probe was larger or smaller than the test disks. They received 120 trials comprising 3 radii of test disks × 2 attended apparent motion paths (left and right diagonal) × 20 replications. 

Results

Figure 5 shows the averaged proportion of trials in which the probe was perceived to be larger than the test. The small graphs on the right side of Figure 5 show individual data. Using the group data, I conducted a two-way ANOVA with the radii of the leading disks and the radii of the test disks as factors. The main effect of the radii of the leading disks was significant, F(1, 4) = 8.270, p < .05. The main effect of the radii of test disks was also significant, F(2, 4) = 23.722, p < .05. Interactions between these two factors were also significant F(2, 4) = 4.526, p < .0005. The simple main effect of the leading disk was significant when the radius of the test was 1.9 or 2.37 deg, Fs(1, 6) = 12.797 and 8.070, respectively, p < .02. 

Figure 5

 

The results of Experiment 3. Error bars denote standard errors of the means. Small graphs show individual data.

Discussion

These results showed that feature attribution occurred in accord with the attended leading disk. This indicates that feature attribution is a higher-order phenomenon that relates to both attention and object representation. Furthermore, the results support a previous finding that a non-retinotopic feature attribution occurs along the attended motion path (Otto et al., 2006). 

General discussion

In the three experiments presented here, I showed that the visual size of a disk is both retinotopically and non-retinotopically attributed and suggested that feature attribution may be a phenomenon that is generalized to visual features other than vernier offset. 

Several studies have indicated that visual features are integrated along the path of movements. Recently, Nishida et al. (2007) showed that color information, for example, red and green, is integrated into a unitary color (yellowish), along a motion trajectory. Moreover, Cavanagh and Holcombe (2005) demonstrated that an attention-based motion could resolve spatial orientation–color binding. As described in the Introduction, a vernier offset is also integrated across lines in an ambiguous apparent motion (Ogmen et al., 2006). In this way, feature integration along a motion trajectory is observed in various types of features, the fact leads to me suggest that integrating visual features across space and time may function to disambiguate and to extract objects from complex and noisy dynamic signals. 

Not only visual features, but also the positions of objects, are integrated across space and time. Roulston, Self, and Zeki (2006) showed that the final location of moving objects that disappeared with a flash was perceived to be behind the flash position. These authors proposed that positional information before the flash is integrated and averaged with weight across space and time, and that this may lead to the perception of a moving object behind the flash position: Their interpretation conforms to previous studies suggesting the existence of a spatial averaging mechanism (Krekelberg & Lappe, 2000; Morgan, 1975). 

I suggest that a similar averaging mechanism works in feature attribution. In Experiments 1 and 2, the perceived size of a test disk was not just identical to the leading disk. This means that pure “attribution” of the size of leading disks to the test disk did not occur. Rather, during the integration of size, the sizes of the leading and the test disks may be averaged. Based on these interpretations, I speculate that averaging is a strategy of the integration process to dynamically prevent discontinuity of the visual world and update our representation of the visual world in a smooth and continuous manner. Given the failure of updating across spatiotemporal changes, no continuous representation of objects is obtained; thus, different objects are perceived across the changes (Moore & Enns, 2004; Moore, Mordkoff, & Enns, 2007). 

I investigated the effect of leading disks on the size of test disk by having observers compare the size of a test disk with that of a probe disk. However, it is possible that the modulated size of the test disk consequently affected the perceived size of the probe disk. Thus, there is a possibility that the observed effect in our study is underestimated due to successive feature attribution in the perception of visual size. Specifically, for example, the small leading disk would lead to the underestimation of the size of the test disk; furthermore, the underestimated size of the test disk may also lead to the underestimation of the probe disk. Future studies should develop a paradigm that can measure feature attribution in the perception of visual size without any influence of the perceived size of a test disk on that of a probe disk. 

The involvement of a motion mechanism should also be discussed since, as shown by the results of Experiment 2 and the additional experiment, ISIs that strongly influence apparent motion seem to be a critical parameter in my study. The present study used a small or large leading disk followed by medium-sized test and probe disks. The successive presentation of these disks yielded the percepts of expanding and contracting motion. Thus, it is possible that motion-tuned neurons (for a description, see Graziano, Andersen, & Snowden, 1994) may be involved in the size modulation effect observed in our study. That is, since the test disk temporally intervened between the leading and probe disks, the size of the test disk might be modulated in accordance with the motion context induced by the leading and probe disks. 

Finally, I would like to speculate about the relationship between a famous geometrical illusion, the Delbouef illusion, and the results of this study. In the Delbouef illusion, a test disk surrounded by a larger inducer disk is perceived as larger than actual (for a recent study, see Surkys, Bertulis, & Bulatov, 2006). Interestingly, several recent studies showed that attending to a task-relevant dimension, that is, the visual area of an outer disk, before the size judgment of the test disk, strengthened the Delbouef illusion (Kawahara, Nabeta, & Hamada, 2007; Hamada, Nishimura, Paramei, & Ehrenstein, 2002). In our study, observers were first presented with either of two leading disks followed by a test disk. Thus, observers likely attended to the leading disk, and this might cause a judgment-order effect similar to that reported by these authors. Furthermore, since only attended leading disks exerted feature attribution, the role of attention in our phenomenon seems to be critical. Further studies are warranted to investigate the role of attention in feature attribution in space and time. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Takahiro Kawabe. 

Email: [e-mail]. 

Address: 6-19-1, Hakozaki, Higashi-ward, Fukuoka, 8128581, Japan.