Segmentation by depth does not always facilitate visual search

Abstract

Abstract

Abstract:

Abstract  In visual search, target detection times are relatively insensitive to set size when targets and distractors differ on a single feature dimension. Search can be confined to only those elements sharing a single feature, such as color (Egeth, Virzi, & Garbart, 1984). These findings have been taken as evidence that elementary feature dimensions support a parallel segmentation of a scene into discrete sets of items. Here we explored if relative depth (signaled by binocular disparity) could support a similar parallel segmentation by examining the effects of distributing distracting elements across two depth planes. Three important empirical findings emerged. First, when the target was a feature singleton on the target depth plane, but a conjunction search among distractors on the nontarget plane, search efficiency increased compared to a single depth plane. Second, benefits of segmentation in depth were only observed when the target depth plane was known in advance. Third, no benefit of segmentation in depth was observed when both planes required a conjunction search, even with prior knowledge of the target depth plane. Overall, the benefit of distributing the elements of a search set across two depth planes was observed only when the two planes differed both in binocular disparity and in the elementary feature composition of individual elements. We conclude that segmentation of the search array into two depth planes can facilitate visual search, but unlike color or other elementary properties, does not provide an automatic, preattentive segmentation.

Introduction

Finding your particular piece of luggage on a crowded baggage carousel is a common experience and one that illustrates the relevance of visual search in daily life. Search can be very efficient when the target bag differs from other distracting bags in a single salient feature, such as color. Indeed, it will likely be identified quickly regardless of the number of distracting bags on the carousel. Such efficient search has been argued to reflect an underlying parallel processing of low-level visual features, in which a unique value on a single feature dimension can be assessed in parallel. In contrast, search for your bag will be inefficient when it cannot be distinguished from other bags by a single feature difference, and the time to find it will be dependent on the number of bags with similar features. For example, if you have marked your bag by attaching a circular red tag, search will be inefficient if others have marked their bags with red and green circular and square tags. In this case, locating the tag requires search of each bag for a conjunction of two features, shape (circular) and color (red). Laboratory experiments analogous to the above example show that conjunction search takes longer and is less efficient in the sense that search time is a function of the number of distractors (Treisman & Gelade, 1980), requiring serial shifts of attention from item to item to identify the target. This search pattern follows from evidence indicating that search along two separate feature dimensions cannot be done in parallel. 

Evidence for parallel search has been found for targets that are singletons in color (Folk, Remington, & Johnston, 1992; Theeuwes, 1992), orientation (Bergen & Julesz, 1983), or size (Treisman & Gormican, 1988). It has also been shown that search can be restricted to items that share the target feature, such as color or orientation (Egeth et al., 1984; Kahneman, 1973). When the target color or orientation is known in advance, the effective set size is reduced to the number of items sharing that feature, suggesting that parallel processing can segment out the items with a particular property across the entire array (Egeth et al., 1984). Curiously, although our visual world is three-dimensional, much less is known about the role of depth information in visual search. Past research has found evidence that singleton targets unique in depth can lead to parallel search (Dunser, Billinghurst, & Mancero, 2008; Nakayama & Silverman, 1986; Steinman, 1987), but there are few studies concerned with determining if segmenting a scene into multiple depth planes enables search to be carried out on one depth plane without interference from other depth planes. This is the focus of the present report. 

Depth segmentation in visual search

Previous studies have shown that attending to items separated across two or more stereoscopic depth planes can facilitate performance. Viswanathan and Mingolla (2002) found improved motion tracking accuracy when search items were presented over two depth planes compared to one (see also Dunser & Mancero, 2009), while Xu and Nakayama (2007) found participants had better memory for target properties when the items were spread over different depth planes. Conversely, Hayes, Moore, and Wong (2006) examined the effect of depth on visual clutter (i.e., the number of display elements in a given display area), finding only small, nonsignificant benefits for 3D displays over 2D displays. Further, Theeuwes, Atchley, and Kramer (1998) showed that target-relevant distractors on an unattended depth plane interfered with search times, despite finding no interference from target-irrelevant distractors. In summary, although segmenting items across two or more depth planes can sometimes be used to increase the efficiency of certain tasks, it is not always possible to ignore or inhibit information from an irrelevant depth plane. 

In one of the few studies examining depth segmentation and visual search, Nakayama and Silverman (1986) showed that conjunction targets differing from distractors in both color and depth could be found efficiently. Search times for targets comprised of a conjunction of motion and color increased linearly with increases in set size. In contrast, search times for color (or motion) targets were insensitive to set size when they differed only in color (or motion) from other elements on their depth plane. Nakayama and Silverman suggested that parallel processing can occur within one depth plane at the same time that information from other depth planes was suppressed. The implications of their results are twofold. First, their work suggests that separation in depth (signaled by binocular disparity) acts similarly to color or orientation in providing a segmentation of the visual display that will support a reduction in the search set. Second, it suggests that such depth segmentation occurs in parallel, similar to the parallel segmentation based on color, orientation, or other elementary features. 

However, Nakayama and Silverman's (1986) design involved depth as a feature of the conjunction search as well as the attribute used to segment the scene. For example, in their color condition the front plane consisted of blue distractors, and the back plane consisted of red distractors. The target was either a red element in the front plane or a blue element in the back plane. In this case, the same feature used to segment items is also one of the defining features of the target, rather than differentiating the conjunction features from the feature used to segment items. It is possible, then, that attention could have been tuned to the feature value (motion direction or color) and the task done, not by searching, but by detecting a depth singleton in one of the two attended feature values. Note that this process would have the effect of elevating search times compared to a single-feature one-plane search, without an effect of set size. This is exactly the result they obtained. 

Nakayama and Silverman (1986) attributed their efficient search results to visual attentional mechanisms that can restrict search to a single depth plane. This observation would imply that the benefits of depth segmentation would apply to displays that did not produce single-feature differences on a given plane. For example, Egeth et al., (1984) found that when a target is denoted by a single color, only distractors of that color determine set size effects. However, Nakayama and Silverman (1986) do not report experiments that would dissociate feature search from efficient search through segmentation. Moreover, Dunser et al. (2008) were unable to replicate the effect of efficient search for a conjunction of color and stereoscopic depth. 

The current study examined if attending to one depth plane would eliminate interference from distractors on a separate depth plane, thus increasing search efficiency. To test this question we created a conjunction search set in which the target was an upright T among Os and tilted Ts, requiring the conjunction of orientation and shape. This search set was then divided into two depth planes, one consisting of tilted Ts, the other Os. We define these as feature planes to denote a set of items with the same binocular disparity that can be distinguished from a target on the basis of a single feature. If attention can successfully eliminate interference from a nontarget depth plane, then search times should resemble those of feature search. Conversely, if items on the nontarget plane cannot be rejected, then search should resemble conjunction, consistent with the combined search set. In Experiments 1 to 3, the search set was distributed so that the target T appeared on the plane with the Os. In Experiments 4 and 5 we address whether search can be confined to one plane more generally, by creating two conjunction planes, in which targets can only be distinguished from distractors by a conjunction of orientation and shape. If so, then search slopes should be the function only of the number of items on the target depth plane. 

Experiment 1

In Experiment 1 we compare search times for targets and distractors presented on a single conjunction plane to search times when the search set is distributed across two feature planes. Participants searched for an upright T among Os and tilted Ts, a conjunction of orientation and letter identity. In the two-depth-plane condition, distractors on the nontarget plane were tilted Ts, while those on the target plane were Os. If attention can be restricted to the target feature plane, then search should be fast and unaffected by set size. If not, distractors on the separate nontarget plane will combine with those on the target plane to create a conjunction search, so that search times will increase with increasing set size. Experiment 1 is similar to Nakayama and Silverman (1986) with the key exception that depth is not one of the properties used to distinguish targets from nontargets. If the findings of Nakayama and Silverman (1986) generalize, participants should perceive this arrangement as distinct searches on separate depth planes and engage in parallel search on the feature plane. 

Method

Participants

Eleven subjects (three female) participated as unpaid volunteers. Stereoacuity was measured using the Titmus Stereo test (Stereo Optical Co., Chicago, IL), and all had acuities of 30 arcsec or less at 16 in., with normal or corrected-to-normal vision. 

Apparatus and stimuli

Stimuli were drawn and scripted using Matlab (MathWorks, Natick, MA) and the Psychophysics Toolbox extension (Pelli, 1997) and presented on two 24 in. Macintosh Cinema displays (1680 × 1050 pixels) in a mirror stereoscope. The two monitors faced each other with mirrors set between and reflecting an image from each monitor to each eye. The viewing distance was 60 cm, and one pixel subtended 1.5 arcmin. Left and right arrow keys of a standard computer keyboard were used for responses. 

The search array consisted of white Os and tilted Ts, arranged in a matrix of five columns × four rows displayed on a black background. In the 3D condition, the items were distributed over two depth planes. In the 2D control condition, all items were presented in a single depth plane. The relative binocular disparity of the two depth planes was 22 arcmin (12 arcmin in front of and 10 arcmin behind the plane of the display). The center-to-center vertical distance between two elements was constant at 1.75°, while their horizontal distances varied randomly between 1.89° and 1.56°. The size of each element was 44 × 44 arcmin. The total number of elements varied depending on set size, among 6, 12, or 18. Half of the elements were white, tilted Ts and the other half were white Os. When present, the target was a white, upright T, replacing one of the Os. The fixation dot was a white disk (4.5 arcmin radius) in the center of the search array. 

Procedure and design

Participants were instructed to make a speeded response to indicate whether an upright T was present or absent by pressing the left or right arrow keys, respectively. On each trial, a fixation dot was displayed at the depth of the target for 400 ms, after which the search display of Os and tilted Ts was presented. On target-present trials (75%) one upright T was presented at the indicated target depth. The search task was displayed until a response was made, at which point a fixation dot turned green if correct or red if incorrect. Additionally, a tone sounded if the response was incorrect. In the one-depth plane (2D) condition, targets and distractor items appeared together on a single depth plane (either the front or the back). For the two-depth-plane (3D) condition, the Os appeared at the target depth and the tilted Ts appeared at the opposite depth plane, creating two feature planes. Participants were informed at the beginning of each condition what the target depth would be, indicated by the words “Front” or “Back” appearing for 2 s. Figure 1 illustrates the search array for the one- and two-depth-plane conditions. 

Figure 1

 

Stereograms and cartoon displays of the experimental stimuli for visual search displays in Experiment 1, where the target is an upright T. The stereograms are for cross-fusion. (A) One-depth-plane condition, and (B) two-depth-plane condition.

Participants completed 720 trials in total, with rests after each set of 180 trials. Target depth and number-of-depth-planes were blocked with block order randomized across participants, and set size and target presence were randomized within each block. There were an equal number of trials for each set size, and the target was present on 75% of trials, with target-absent trials acting as catch trials. In the two-depth-planes condition, there were an equal numbers of items at each depth. 

Results

Mean accuracy was 96%. All error trials were removed from the analysis of response times. In the two-depth-plane condition, mean response times did not differ as a function of whether the target was in the front or back depth plane, t(10) = 0.54, p = 0.601. Therefore subsequent analyses collapsed the target-front and target-back conditions. 

Figure 2 plots mean response times as a function of set size for one-depth-plane and two-depth-plane conditions. A two (number-of-depth-planes: 1, 2) by three (set size: 6, 12, 18) repeated-measures ANOVA conducted on the target-present data revealed a main effect of number-of-depth-planes, F(1, 10) = 24.02, p = 0.001, and set size, F(1, 10) = 41.83, p < 0.001, as well as a significant interaction between number-of-depth-planes and set size, F(2, 20) = 12.97, p < 0.001. We followed up the interaction by comparing the search slopes for one-depth-plane and two-depth-planes conditions. To calculate search slopes for each participant, response times for set size 6 were subtracted from those at set size 18, and divided by the difference in number of items (18 − 6 = 12). A paired-sample t test revealed that the search slope for the two-depth-planes condition (M = 4.92) was significantly shallower than the search slope for one-depth-plane condition, M = 13.41, t(10) = 3.91, p = 0.003. 

Figure 2

 

Mean response times for target-present data as a function of set size and number of depth planes in Experiment 1. Search slope values are given in brackets (ms/item). Error bars show standard error.

Discussion

The results from Experiment 1 clearly demonstrate that participants found targets faster and more efficiently when items in the search array were segmented by depth into two feature planes compared to one depth plane. Consistent with previous results (Dent, Braithwaite, He, & Humphreys, 2012), we found no response time differences when presenting the target on the front depth plane versus the back depth plane. 

Experiment 1 used a large disparity to separate the two depth planes, and it is possible that items on the nonfixated plane fell outside Panum's fusional range and appeared diplopic (perceived doubling of the image) (Grove, 2012; Howard, 2012). This disparity may have reduced the degree to which items on the nonfixated plane interfered with the target on the other plane. In Experiment 2 we repeated our measurements for several disparities to control for this possibility. Furthermore, Experiment 2 followed up a report by de la Rosa, Moraglia, and Schneider (2008), who investigated the minimum disparity between depth planes eliciting efficient search. Using the efficient conjunction search reported by Nakayama and Silverman (1986) for depth and color, they found that rapid search occurs optimally with disparities of 6 arcmin or more, noting that at smaller separations it may be the case that information from one depth plane sometimes intrudes upon another. However, like Nakayama and Silverman (1986), they used disparity as one of the features in the conjunction search. In Experiment 2 we tested the effect of the magnitude of disparity on search performance using the conditions of Experiment 1 in which the conjunction is formed by the combination of two depth planes without depth per se being a defining feature of the target. 

Experiment 2

Method

Participants

Fifteen subjects (seven females) participated as unpaid volunteers. Stereoacuity was measured using the Titmus Stereo test (Stereo Optical Co., Chicago, IL), and all had acuities of 30 arcsec or less at 16 in., with normal or corrected-to-normal vision. 

Apparatus and stimuli

Except as noted, the stimuli and displays were as described in Experiment 1. There were six different disparities separating the two depth planes: 0, 1.5, 3, 6, 9, and 12 arcmin between the front and back depth plane. The 0 arcmin separation, the single depth plane condition, presented items in the plane of the display. The 1.5 arcmin condition presented half the items on the back depth plane with 1.5 arcmin uncrossed disparity and half the items on the front plane at 0 arcmin. For all remaining disparity conditions, half the magnitude of the disparity was introduced as uncrossed disparity, and the other half as crossed disparity relative to the plane of the display, with half the items presented in each depth plane. 

Procedure and design

In Experiment 1 we found no differences in search performance between front and back target depth, so we always presented the target in the front depth plane. Accordingly, participants were told that the target, when present, would always appear in the near depth plane. Trial sequence and timing were also the same as in Experiment 1. 

Participants completed 1,080 trials in total, blocked across the six levels of disparity, with rests after each set of 180 trials. Block order was randomized across participants, and set size and target presence were randomized within each block. There were an equal number of trials in each disparity condition. 

Results

Participants had a mean accuracy of 96%, and all error trials were removed from the analysis of response times. Search slope values were calculated across set sizes as in Experiment 1, and are shown in Figure 3. A six (disparity between depth planes: 0, 1.5, 3, 6, 9, 12 arcmin) by three (set size: 6, 12, 18) repeated-measures ANOVA was conducted on the target-present data. We found a significant main effect of set size, F(2, 28) = 48.54, p = < 0.001, and disparity, F(5, 70) = 13.98, p = < 0.001, as well as a significant interaction between disparity and set size, F(10, 140) = 6.30, p = < 0.001. We followed up the main effect of disparity with paired sample t tests with a Bonferroni correction for five comparisons (α = .01). The only significant difference was found for mean response times between 0 arcmin disparity (M = 684.57) and 1.5 arcmin disparity response times, M = 606.82, t(14) = 3.21, p = 0.006, with no other significant differences. We calculated search slopes for each level in the disparity between depth planes condition, and followed up the interaction with paired sample t tests, using a Bonferroni correction for five comparisons (α = 0.01). Again, the 0 arcmin disparity search slope (M = 12.59) was significantly steeper than the 1.5 arcmin disparity slope, M = 5.40, t(14) = 3.82, p = 0.002, with no other significant differences, 1.5 arcmin = 3 arcmin, t(14) = −1.18, p = 0.259; 3 arcmin = 6 arcmin, t(14) = 0.85, p = 0.412; 6 arcmin = 9 arcmin, t(14) = 1.19, p = 0.253; and 9 arcmin = 12 arcmin, t(14) = −0.05, p = 0.963. 

Figure 3

 

Search slope values for each depth-plane separation disparity, for target present data. The dotted lines represent the search slopes for the one-depth-plane (upper) and two-depth-planes (lower) conditions from Experiment 1. Error bars show standard error.

Discussion

The results from Experiment 2 replicated the increased search efficiency found for targets in Experiment 1 when search items were spread over two feature planes compared to one depth plane. In addition, our data indicated that the advantage of search across multiple depth planes occurred even at very small separations between depth planes, as search efficiency did not vary with different nonzero disparity magnitudes. Even the search slope for the smallest separation of 1.5 arcmin reached similar levels of efficiency to those seen in Experiment 1, which used a separation of 22 arcmin. This indicates that even for disparities well within Panum's fusional range (Grove, 2012; Howard, 2012), organizing features into separate depth planes is possible and facilitates search efficiency. As such, this search improvement cannot be attributed to diplopia reducing the saliency of the distractors in the nonfixated depth plane. Furthermore, response times and search slopes both remained constant from the smallest to the largest disparities used, and did not reduce in difficulty or increase in efficiency beyond the smallest separation disparity. This finding contrasts with the findings of de la Rosa et al. (2008), who found that separating depth planes by disparities less than ∼ 6 arcmin resulted in information from one depth plane sometimes intruding upon another depth plane. One difference between our study and theirs is the magnitude of simulated depth. De la Rosa and colleagues simulated depth ranging from 1.32 to 22.4 cm, whereas our experimental range was only 0.72 to 5.8 cm. While we do not fully understand the reasons for these contrasting results, unlike de la Rosa et al. (2008 and Nakayama and Silverman (1986), we used disparity to segment the two displays and not as a defining feature of the conjunction target. It is possible, then, that when depth is a defining target, feature variations in the magnitude of that feature directly affects the difficulty of target-nontarget discrimination. 

Experiment 3

Experiments 1 and 2, as well as Nakayama and Silverman (1986), all included explicit instructions regarding the depth of target items, or required attention to item disparity to complete the task. One of the implications of Nakayama and Silverman's (1986) results is that, like color or orientation, depth facilitates a preattentive segregation of items. This facilitation raises the question of what role attention plays in search on discrete depth planes. If depth information is available preattentively and it is possible to detect the target by a parallel search of multiple depth planes, then it should not be necessary to know in advance which depth plane contains the target. In Experiment 3 we examined the role of withholding explicit target depth instruction in searching two depth planes, to see if knowledge of target depth is a necessary condition for the improved search efficiency demonstrated in Experiments 1 and 2. In this experiment, participants searched for a target among distractors spread over one or two depth feature planes (as in Experiment 1), but this time the target depth plane varied trial to trial, and participants were not informed about which depth plane the target would appear in. 

Method

Participants

Sixteen subjects (nine female) participated as either unpaid volunteers or from the first-year participant pool at the University of Queensland, Australia, receiving course credit for participation. Stereoacuity was measured using the Titmus Stereo test (Stereo Optical Co., Chicago, IL), and all had acuities of 50 arcsec or less at 16 in., with normal or corrected-to-normal vision. 

Apparatus and stimuli

Except as noted, the stimuli and displays were as described in Experiment 1. When there was one depth plane, all items were presented in the plane of the display. When there were two depth planes, the target was randomly positioned on either the front or the back depth plane. As in Experiment 1, for the two-depth-planes condition the Os appeared at the target depth and the tilted Ts appeared at the opposite depth plane, creating two feature planes. Binocular disparity separating the two depth planes was the same as Experiment 1. 

Procedure and design

Participants received similar instructions to Experiment 1, except that they were told they would not know which depth plane the target would appear in. Trial sequence and timing were also the same as Experiment 1. 

Participants completed 540 trials, blocked across number-of-depth-planes, with rests after every 180 trials. Block order was randomized across participants, with set size and target presence randomized within each block. Of the total trials, 360 were two-depth-planes and 180 were one-depth-planes. 

Results

Participants had a mean accuracy of 97%, and all error trials were removed from analysis. In the two-depth-plane condition, mean response times did not differ as a function of whether the target was in the front or back depth plane, t(15) = 1.86, p = 0.082. Therefore subsequent analyses collapsed the target-front and target-back conditions. 

Figure 4 plots mean response times for one- and two-depth planes as a function of set size. A two (number-of-depth-planes: 1, 2) by three (set size: 6, 12, 18) repeated-measure ANOVA conducted on the target-present data revealed a significant main effect of set size, F(2, 30) = 60.41, p < 0.001, with no other significant main effects or interactions. Critically, there was no difference found between the search slopes for when the items were on one-depth-plane or two-depth-planes, t(15) = 1.74, p = 0.103. 

Figure 4

 

Mean response times for target-present data as a function of set size and number of depth planes in Experiment 3. Search slope values are given in brackets (ms/item). Error bars show standard error.

Discussion

The results of Experiment 3 showed that without prior knowledge of which depth plane contained the target, search across two depth planes was no more efficient or faster than search within one depth plane. Knowledge of the target depth plane is necessary to facilitate the detection of feature singleton targets even when the target was a feature singleton on its depth plane. This finding indicates that depth segmentation is not used automatically, but requires that attention be directed at a depth plane for parallel search of the plane. 

Experiment 4

In Experiments 1–3 we extended the previous work of Nakayama and Silverman (1986) by providing evidence of parallel search when the target-defining feature is distinct from the depth information used to segment the display. Further, we showed that multiple depth planes could not be searched in parallel. There remains the question of whether depth information acts like features of individual objects in restricting search sets, as implied in the account of Nakayama and Silverman (1986). For example, Egeth et al., (1984) found that when a target was denoted by a single color, only distractors of that color determined set size effects. In Experiments 1 and 2, search slopes in the two-depth-plane conditions were roughly half that of the one-depth-plane condition. This is what would be expected if the depth information, without prior knowledge of the target, allowed participants to search only half the items. We addressed this effect more directly in Experiment 4 by investigating if distributing the search set across two depth planes per se facilitates search when the target and nontarget planes are both conjunction planes. If subjects could restrict search to the target depth plane, as Egeth et al. (1984) found for color, then slopes should be roughly half for the two-depth-plane condition than in the one-depth-plane condition. 

Method

Participants

Seventeen subjects (11 female) participated as either unpaid volunteers or from the first-year participant pool at the University of Queensland, Australia, receiving course credit for participation. Stereoacuity was measured using the Titmus Stereo test (Stereo Optical Co., Chicago, IL), and all had acuities of 50 arcsec or less at 16 in., with normal or corrected-to-normal vision. 

Apparatus and stimuli

Except as noted, the stimuli and displays were as described in Experiment 1. In addition to the one-depth-plane and two-depth-plane conditions previously described (see Figure 1), a third condition was added where the items were spread over two depth planes, pseudo-randomly allocating each O and tilted T to either the front or back depth plane to create two conjunction searches (conjunction planes) (see Figure 5). In this condition, each depth plane contained half of the items with an equal number of Os and tilted Ts. Target depth was blocked within each depth-plane condition. 

Figure 5

 

Stereogram and cartoon display of the experimental stimuli for visual search displays in Experiment 4, where the target is an upright T. The stereograms are for crossed-fusion.

Procedure and design

Participants received the same instructions as in Experiment 1. Trial sequence and timing were also the same as in Experiment 1. Participants completed 864 trials, blocked across both depth-arrangement condition and target depth, with rests after every 144 trials. Block order was randomized across participants. Set size and target presence were randomized within each block. Trials were divided equally among the three depth plane conditions. 

Results

Participants had a mean accuracy of 96%, and all error trials and trials with response times greater than three standard deviations above the overall mean for each subject were removed from analysis. In the two-depth-plane condition, mean response times did not differ as a function of whether the target was in the front or back depth plane, t(16) = 1.50, p = 0.269. Therefore, subsequent analyses collapsed the target-front and target-back conditions. 

Figure 6 plots mean response times for each depth arrangement condition as a function of set size. A three (depth arrangement condition: one-depth-plane, two-depth-planes conjunction, two-depth-planes feature) by three (set size: 6, 12, 18) repeated-measures ANOVA on the target-present data revealed a significant main effect of depth arrangement condition, F(2, 32) = 8.02, p = 0.002, and set size, F(2, 32) = 65.72, p < 0.001, as well as an interaction, F(4, 64) = 4.02, p = 0.004. The main effect of depth-arrangement condition was followed up with paired sample t tests, finding that the two-depth-planes feature condition was significantly different from both the two-depth-planes conjunction condition, t(16) = 2.81, p = 0.012, and the one-depth-plane conjunction condition, t(16) = 3.81, p = 0.001, with no significant difference between one-depth-plane and two-depth-planes conjunction conditions, t(16) = 0.73, p = 0.473. Paired sample t tests on the individual search slopes revealed that the two-depth-planes feature slope was significantly shallower than the two-depth-planes conjunction slope, t(16) = 2.56, p = 0.021, and the one-depth-plane conjunction slope, t(16) = 2.42, p = 0.028, whereas there was no difference between one-depth-plane and two-depth-planes conjunction slopes, t(16) = 0.45, p = 0.660. 

Figure 6

 

Mean response times for target present data as a function of set size and depth-plane condition in Experiment 4. Search slope values are given in brackets (ms/item). Error bars show standard error.

Discussion

Experiment 4 replicated our findings from Experiment 1 for the two-depth-plane feature search condition, showing increased speed and efficiency when depth segmentation creates a target feature plane. However, depth segmentation did not aid search when both planes were conjunction planes compared to the one-depth-plane condition. There was no evidence that participants could restrict the serial search to a single depth plane, despite prior knowledge of the target depth plane; instead, set size was determined by distractors on both planes. The pattern for the conjunction search is similar to findings by Theeuwes et al. (1998), who showed that relevant distractors on different depth planes captured attention even when participants knew the target would be on a different plane. 

Experiment 5

Although Experiment 4 found no difference between the one-depth-plane and two-depth-plane conjunction conditions, our search slopes are considerably shallower than typically found in conjunction searches (e.g., Nakayama and Silverman, 1986). Our conjunction of tilt and shape may have left open the possibility that subjects were not using depth segmentation to isolate the target plane, but instead using the strong feature difference between Os and Ts to segment the display into depth planes. It is also possible that parallel search occurred in all conditions and that set size effects index distractor noise, rather than serial search. In Experiment 5 we used a more difficult conjunction task to investigate whether our results generalize to more demanding search sets. 

Method

Participants

Twenty subjects (12 female) participated from the first-year participant pool at the University of Queensland, Australia, receiving course credit for participation. Stereoacuity was measured using the Titmus Stereo test (Stereo Optical Co., Chicago, IL), and all had acuities of 50 arcsec or less at 16 in., with normal or corrected-to-normal vision. 

Apparatus and stimuli

Except as noted, the stimuli and displays were as described in Experiment 4. The search array matrix remained the same; however, the search elements were short and long white bars tilted 26.6° to the left or right of vertical. The size of each short element was 17 arcmin × 3 arcmin, and each long element was 27 arcmin × 3 arcmin. The total number of elements varied depending on set size, either 8 or 18. Half of these were short and half were long. When present, the target was a short left-tilting bar, replacing one of the short right-tilting bars. 

Procedure and design

Participants received the same instructions as in Experiment 4, except they were told to search for a short left-tilting bar instead of a T. Trial sequence and timing were also the same as in Experiment 4. Participants completed 480 trials, blocked across both depth-plane conditions and target depth, with rests after every 80 trials. Block order was randomized. Set size and target presence were randomized within each block. Trials were divided equally among the three depth arrangement conditions. In the one-depth-plane condition, the target and distractor items were all presented in the same depth plane as a conjunction search. In the two-depth-plane conjunction condition, the distractors were pseudorandomly assigned half to each depth plane, creating two conjunction planes. In the two-depth-plane feature condition, the short right-tilting bars appeared at the target depth and the long left- and right-tilting bars appeared at the opposite depth plane, creating two feature planes. Figure 7 illustrates the search arrays displayed for each of the three depth arrangement conditions. 

Figure 7

 

Experimental stimuli for visual search display, where the target is a short left-tilting bar. (A) One-depth-plane condition. (B) Two-depth-plane conjunction condition. (C) Two-depth-plane feature condition. Frames are not present in actual experimental display.

Results

Participants had a mean accuracy of 96%, and all error trials were removed from analysis. In the two-depth-plane condition, mean response time did not differ as a function of whether the target was in the front or back depth plane, t(19) = 1.70, p = 0.105. Therefore, subsequent analyses collapsed the target front and target back conditions. 

Figure 8 shows mean response times for target present trials for the three depth arrangement conditions as a function of set size. A three (depth arrangement condition: one-depth-plane, two-depth-planes conjunction, two-depth-planes feature) by two (set size: 8 and 18) repeated-measures ANOVA on the target-present data revealed a significant main effect of depth arrangement condition, F(2, 38) = 7.71, p = 0.002, and set size, F(1,19) = 87.68, p < 0.001, as well as a depth arrangement condition by set size interaction, F(2,38) = 12.04, p < 0.001. The main effect of depth arrangement condition was followed up with paired sample t tests, finding that the two-depth-planes feature condition was significantly different from both the two-depth-planes conjunction condition, t(19) = 3.30, p = 0.004, and the one-depth-plane conjunction condition, t(19) = 3.33, p = 0.003, with no significant difference between one-depth-plane and two-depth-planes conjunction conditions, t(19) = 1.59, p = 0.129. Paired sample t tests revealed that the two-depth-planes feature slope was significantly shallower than the two-depth-planes conjunction slope, t(19) = 4.94, p < 0.001, and the one-depth-plane conjunction slope, t(19) = 4.04, p = 0.001, whereas there was no difference between one-depth-plane and two-depth-planes conjunction slopes, t(19) = 1.24, p = 0.228. 

Figure 8

 

Mean response times for target-present data as a function of set size and depth-plane condition in Experiment 5. Search slope values are given in brackets (ms/item). Error bars show standard error.

Discussion

The results of Experiment 5 replicated Experiment 4 in finding that search for a conjunction target with distractors segmented over two depth planes was no more efficient or faster than when all items were on a single depth plane. The conjunction of orientation and length was successful in creating a more difficult search task, evidenced by the elevated response times relative to Experiments 1–4. As the pattern of results did not change from those observed in Experiment 4, we can conclude that previous results for a conjunction search distributed across depth planes generalizes to more difficult conjunction searches. We again found improved search efficiency for the feature singleton target compared to the conjunction target. However, the search slope was steeper than in previous experiments and too great to easily attribute the effect of set size to parallel search. This result suggests that although the feature singleton was found more efficiently, there was still some interference from distractors on the nonfixated depth plane. We compared search times for the nine items on the target plane in the two-depth-plane feature condition with search times for the eight items on the target plane in the one-depth-plane condition. There was a nonsignificant tendency towards longer response times in two-depth-plane feature condition with nine items (M = 1632 ms), compared to the equivalent single depth-plane conjunction condition with a set size of eight (M = 1435 ms), which is further evidence of interference from distractors on the unattended depth plane. 

General discussion

The present study examined visual search in 3D space with the goal of determining under what conditions segmenting items by depth plane facilitated visual search. Nakayama and Silverman (1986) documented improved search efficiency for a conjunction of stereoscopic depth and color or motion, suggesting parallel search on one of two depth planes. We investigated whether this efficient search was due to feature search within one depth plane at a time, or the effective halving of the set size through segmenting items across two depth planes. This study revealed three important empirical findings. We discuss each finding and its associated implications in turn below. 

First, consistent with Nakayama and Silverman (1986), we found evidence of parallel search when the target and distractors on the attended depth plane could be distinguished using a single feature dimension, even when distractors on the nontarget plane, when added to those on the target depth plane, created a conjunction search (Experiment 1). This effect was replicated and extended in Experiment 2, showing that efficient search is possible even with very small disparities separating the depth planes. The evidence for parallel processing of items on the attended depth plane extends the work of Nakayama and Silverman (1986) to situations where the conjunction features of the target are independent of the segmenting feature. The implication of our finding is that directing attention to a single depth plane triggers parallel processing of items on that depth plane. 

However, if processing of items on the attended depth plane were done in parallel, then the search slopes in Experiment 5 should have approached zero, as in Experiment 1, or been much shallower than the 38 ms/item observed. One possible explanation is that processing of distractors on the nontarget depth plane was attenuated, but not fully inhibited. Even if search were parallel, the added perceptual noise from the increased number of distractors with larger set sizes would be expected to delay target identification, producing a set-size effect. In Experiment 5, where the conjunction task was very difficult, this noise would be more pronounced than in the less difficult task of Experiment 1. Alternatively, we note that all the feature-search conditions had search slopes of about half that of the single plane. The ratio of slopes is what one would expect from a serial, self-terminating search through half the items. We return to this point in discussing the conjunction search conditions. 

The second major finding is that enhanced search efficiency is conditional upon prior knowledge of the depth plane containing the target (Experiment 3). Creating a feature search on the target depth plane only reduced search times when participants knew which of the two planes would contain the target. This finding is contrary to that observed by Nakayama and Silverman (1986), who showed flat search slopes even when participants did not know which of two depth planes would contain the target. Nakayama and Silverman's finding could be taken as evidence that the assignment of features to depth planes occurred preattentively, given that the conjunction of depth with color or motion could be found quickly with no effect of set size in the absence of prior knowledge of the target depth plane. Our results, however, suggest this is not the case, and evidence of parallel search was found only for the attended depth plane. We believe the explanation for the different pattern of results lies in differences in method that lead to strategic adaptation to task demands. In Nakayama and Silverman's color conditions, blue always occurred on the front plane, red on the fixation plane. Had participants chosen to attend to the fixation plane (red), they could have detected the presence of a blue color singleton on that plane, consistent with our findings. Had the target been a red singleton on the front plane, this could have been detected in one of three ways: by switching attention to the front plane and engaging in a parallel search, by attending to red and detecting a depth singleton, or by the absence of a blue singleton on the red plane. All three possibilities are consistent with our observation that parallel search is possible only on the attended plane. Present evidence is insufficient to rule out any of these accounts. 

The implication of our findings is that stereoscopic depth cues are not equivalent to color, motion, orientation, or other features that do produce an automatic segmentation. Attending to color, for example, has been shown to produce an array-wide set for red, evidenced by altered firing rates of color-selective neurons (Bichot, Rossi, & Desimone, 2005). This outcome likely underlies the finding that search can be restricted to items of a feature matching the attentional set (Egeth et al., 1984). However, Experiment 3 showed that knowledge of the target depth plane is necessary to benefit from restricting search to one depth plane. Additionally, if segmenting the display into two depth planes facilitates search by halving the effective set size of each conjunction search, this modification should also reduce search times. However, Experiments 4 and 5 found no search benefits when the items segmented into two depth planes were not two feature planes. 

Our third and perhaps most surprising result is that we found no evidence that segmenting items across two depth planes benefits search for a conjunction target (Experiments 4 and 5). If search could be automatically restricted to one set of items defined by disparity, as seen for color (Egeth et al., 1984), then conjunction search slopes in the two-depth-plane condition should have been half that in the one-depth-plane condition. Instead, they were virtually identical. One explanation for why distributing items across two conjunction planes did not facilitate target detection is that the attentional resources needed to maintain attention on a depth plane are the same as those required for a serial search. Separating items into depth planes may enable information accrual along one feature dimension on the selected depth plane without much interference from distractors in the other depth plane, but not when a conjunction of two features are required to discriminate the target. This is the pattern that would result if we assume that the act of attending to a depth plane draws on the same attentional processes required for target selection within a depth plane, simultaneously attending to one depth plane while trying to select the target. The attentional demands of feature search are minimal, whereas search for a conjunction target are resource demanding. We speculate that due to shared but limited resources, when observers are holding their attention at one depth plane and searching through it, they have fewer resources left over to ignore distractors on the other depth plane. For a more difficult conjunction search, observers' attentional resources are stretched even further when filtering out noise from distractors on the irrelevant depth plane. It is possible that we failed to find a benefit of depth segmentation in the conjunction search task because, in the absence of distinct surfaces, our depth manipulation did not allow participants to effectively group the search elements into two, well-defined depth planes. This account is unlikely given the benefit of depth segmentation in the feature search task, suggesting that participants could search one depth plane while ignoring items of the second irrelevant depth plane. Additionally, participants' feedback during instruction and informal debriefings indicated they clearly perceived two depth planes. 

Many models of visual search classically describe a two-stage process, such as Treisman's Feature Integration Theory (Treisman & Gelade, 1980), which proposes a parallel, preattentive first stage and a serial, attentive, second stage. Wolfe's Guided Search 4.0 (Wolfe, 2007) is similar to Treisman's theory except both stages are described as mostly parallel. Similarly, Signal Detection Theory models of visual search posit a parallel, unlimited-capacity first stage followed by a simple decision rule (Palmer, 1998; Verghese, 2001). Despite their specific differences, these models agree on the basic processes of visual search: visual information is input, information about each item accumulates towards a criterion threshold, an item is selected once it reaches this threshold, the selected item is identified and matched to target specifications, and a response is selected and executed. Recently, Finlayson, Remington, and Grove (2012) proposed that search slopes arise from the addition of decision noise to the first stage of this process, the accumulation of information towards the criterion threshold. The greater the noise in this first stage, the longer it takes the target to reach the threshold for selection. Our results suggest that segmenting the display into two depth planes enables suppression of noise from distractors at other depths only when information about a single feature is being accumulated within this first stage and observers have foreknowledge of the target depth plane. When finding the target requires accumulation of information about two features, less resources are available to inhibit noise from distractors at the other depth planes, which then intrudes upon the accumulation process. Following from this assumption, and in line with Theeuwes et al., (1998), we conclude that depth information from binocular disparity is available early in processing to facilitate goal-directed guidance of attentional selection. 

Conclusion

To the best of our knowledge, no other study has investigated and demonstrated the ability to perform a feature search among items that project as a conjunction search on the retina, but when viewed stereoscopically, appear to occupy two planes segmented in depth. Critically, we dissociated between efficient search due to feature search within one depth plane, and the effective halving of the set size through segmenting items across two depth planes. Our novel findings suggest that selecting one of two depth planes requires attention, meaning that temporary suppression of the nonfixated depth plane can only be maintained without other demands on attentional resources. As such, this benefit is only present during search for feature-singleton targets and not conjunction targets. Advance knowledge of the target depth plane is also required, without which, depth information from binocular disparity is not automatically utilized to facilitate search efficiency. 

Acknowledgments

Supported by an Australian Postgraduate Award (APA) scholarship. Australian Research Council grant DP120103721. 

Commercial relationships: none. 

Corresponding author: Nonie J. Finlayson. 

Email: [e-mail]. 

Address: School of Psychology, The University of Queensland, St. Lucia, Australia.