Orientation anisotropies in visual search revealed by noise

Abstract

The human visual system is remarkably adept at finding objects of interest in cluttered visual environments, a task termed visual search. Because the human eye is highly foveated, it accomplishes this by making many discrete fixations linked by rapid eye movements called saccades. In such naturalistic tasks, we know very little about how the brain selects saccadic targets (the fixation loci). In this paper, we use a novel technique akin to psychophysical reverse correlation and stimuli that emulate the natural visual environment to measure observers' ability to locate a low-contrast target of unknown orientation. We present three main discoveries. First, we provide strong evidence for saccadic selectivity for spatial frequencies close to the target's central frequency. Second, we demonstrate that observers have distinct, idiosyncratic biases to certain orientations in saccadic programming, although there were no priors imposed on the target's orientation. These orientation biases cover a subset of the near-cardinal (horizontal/vertical) and near-oblique orientations, with orientations near vertical being the most common across observers. Further, these idiosyncratic biases were stable across time. Third, within observers, very similar biases exist for foveal target detection accuracy. These results suggest that saccadic targeting is tuned for known stimulus dimensions (here, spatial frequency) and also has some preference or default tuning for uncertain stimulus dimensions (here, orientation).

Introduction

Studies of visual acuity and contrast sensitivity in humans have shown an unequal sensitivity across orientation; generally, these studies find a greater sensitivity to gratings with cardinal (horizontal/vertical) relative to oblique orientations (Berkley, Kitterle, & Watkins, 1975; Campbell, Kulikowski, & Levinson, 1966). This orientation anisotropy is referred to as the “oblique effect” or “cardinal bias” in the literature (Appelle, 1972). This effect is also found in cats and macaque monkeys, but not as consistently as for humans (Li, Peterson, & Freeman, 2003). Some have argued that using broadband stimuli, such as 1/f noise or natural images filtered in orientation, instead of gratings, could instead yield greater sensitivity for oblique orientations (Hansen & Essock, 2004). 

A number of single-cell physiological studies (see Li et al., 2003) of the primary visual cortex (V1) have found variability in populations of orientation tuned cells (viz., a larger number of cells tuned to horizontal and vertical than to oblique orientations), but several other studies were unsuccessful in finding such differences (Finlay, Schiller, & Volman, 1976; Mansfield, 1974). It has been suggested that causes of these discrepancies are that different studies have used different measurement procedures, and that some have sampled only a small population of cells. More recently, researchers have observed an oblique effect using optical imaging (Coppola, White, Fitzpatrick, & Purves, 1998) and functional magnetic resonance imaging (Furmanski & Engel, 2000). Overall, the belief in a generic deficit for oblique stimuli remains a source of some contention, and its incidence is likely to vary with the stage in the visual pathway measured and the experimental technique employed. 

A question of great interest is how visual search might be affected by anisotropies in the perception of orientation. Studies have reported search asymmetries in tasks where human observers seek an oriented target amongst a set of distracters (Carrasco, McLean, Katz, & Frieder, 1998; Foster & Ward, 1991; Treisman & Gormican, 1988; Wolfe, 1998; Wolfe, Friedman-Hill, Stewart, & O'Connell, 1992). For instance, the detection of a tilted line amongst vertical lines has been found to be easier than search for a vertical line amongst tilted lines. In this paper, we address a more general problem in visual search where the orientation of the target is not known to the observer a priori. Such an experimental procedure is similar to many real-world search tasks, in which the orientation of an object is largely uncertain, though it may be influenced by gravity or its proximal interaction with other objects and planes. We use a new and efficient experimental search framework (Tavassoli, van der Linde, Bovik, & Cormack, 2007), extending earlier techniques (Ahumada, 1996; Eckstein, Beutter, Pham, Shimozaki, & Stone, 2007; Rajashekar, Bovik, & Cormack, 2006), to study the behavior of humans seeking a randomly oriented grating embedded in noise with an amplitude spectrum closely resembling that found in images of natural scenes (Field, 1987). 

Methods

Observers

Four male observers (aged 26 through 30), of whom two were experienced (authors A.T. and I.V.D.L.) and two naive (A.E.P. and A.J.S.), were tested in our experiments, each with normal/corrected-to-normal vision. Each observer completed 1,400 trials (2 sets of 700 trials, separated by a period of about 1 month). 

Stimuli

Our search target was a 64 × 64 pixel Gabor patch of frequency 8 cycles/deg and bandwidth 0.25 octaves ( Figure 1a). One hundred 7 × 7 tile mosaics were generated offline by creating one hundred 544 × 544 pixel 1/ f noise images (with an amplitude spectrum of the form 1/ f ^a with a = 0.8) and then superimposing gray borders 12 pixels in width ( Figure 1c). On each trial, the orientation of the Gabor was randomly selected from the set {0°, 1°, 2°,… 179°} ( Figures 1a and 1b) and this Gabor was then added to a randomly selected tile of the 1/ f noise grid ( Figure 1c). As a convention, angles ascended from 0° (vertical bars) in an anticlockwise direction. Observers viewed the stimuli on an Image Systems 21-in. grayscale monitor (Image Systems Corp., Minnetonka, MN) driven by a Matrox Parahelia graphics card (Matrox Graphics Inc., Dorval, Québec, Canada) at a screen resolution of 1,024 × 768 pixels, a grayscale resolution of 8 bits per pixel, and a refresh rate of 60 Hz. The screen was placed 134 cm from the observer and subtended a visual angle of 16° × 12°, giving approximately 1 min of arc per screen pixel. The luminance output was linearized by putting the inverse of the monitor's measured gamma function in the display look-up table. The ambient illumination in the laboratory was kept constant for all observers, and there was a minimum of 5 min to adapt to the ambient illumination and screen luminance while the eye tracker was calibrated. 

Figure 1

 

Stimulus creation, data capture, and data analysis. (a) A Gabor patch was used as a target and (b) its orientation was randomly selected from the set {0°, 1°, 2°,… 179°}. (c) The target was added to a randomly selected tile of the 1/ f noise grid and observer eye movements were recorded while they searched for the target. An example of scan path is shown for a trial in which the observer did not find the target, located in the center of the leftmost column. (d) Fixated tiles that did not contain the target constitute our nonfoveal false alarm category, and (e) a subset of these tiles, which were mistakenly selected at the end of trials as the target by the observer, constitute our foveal false alarm category. (f and g) Average difference spectra were computed by averaging the amplitude spectra of noise tiles in each category and subtracting the spectral bias (see text).

Procedure

Eye movements were recorded while observers searched the stimulus grid for the Gabor target. An SRI/Fourward Generation V Dual Purkinje eye tracker (Fourward Technologies Inc., Buena Vista, VA) was used to record eye movements. Stimuli were presented for 5 s on the calibrated grayscale monitor using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). Observers were required to fixate a central fixation mark to initiate each trial. Observers then searched the stimulus grid and maintained their final fixation on the tile they believed to contain the target (over 81% of the dwell times observed for final fixations were equal to or longer than 600 ms, an upper bound on typical fixation durations, indicating that observers were deliberately selecting a single tile as containing the target on most trials). The signal-to-noise ratio of stimuli was adjusted for each observer (i.e., we covaried the contrast of the target and of the noise such that the entire grayscale was used but never exceeded) using the QUEST adaptive procedure (Watson & Pelli, 1983) to yield an average correct target detection rate close to 68%. 

Analysis method

During each trial, observers would perform four to five fixations on average therefore visiting tiles not containing the target (i.e., noise-only tiles), and in some trials selecting one such tile as the target; an example stimulus grid with overlaid eye movements for a single observer is shown in Figure 1c. Thus, we asked two questions. First, why were some noise-only tiles fixated whereas the others were not? And second, why, at the end of some trials, was a noise-only tile mistakenly selected as the tile containing the target? 

To answer these questions, we assume that each fixation (excluding the initial fixation at stimulus onset) involves two decisions: the decision to fixate a certain tile (and not the others) and the subsequent decision to either remain on that tile or continue searching. We consider that the former is based primarily on nonfoveal information and the latter is based primarily on foveal information. We therefore stored noise-only tiles that were fixated while en route to the target and labeled them as “nonfoveal false alarms” (

$\bar{f}$

_FA; Figure 1d). Additionally, noise-only tiles that were mistakenly selected as the target at the end of a trial were labeled as “foveal false alarms” ( f _FA)—these necessarily being a subset of the nonfoveal false alarms ( Figure 1e). These signal-absent categories better reflect observer behavior than signal present categories (those composed of tiles that contained the target) because only patterns in the noise, corresponding to visual information that the observer took to imply the presence of a target, are used (Eckstein, Shimozaki, & Abbey, 2002). Further information on the employed taxonomy may be found in our previous work (Tavassoli et al., 2007). 

We then computed the Fourier transform of each tile and averaged their amplitude spectra within category and observer. Because we used a finite number of 1/ f noise tiles (100 × 7 × 7 = 4,900) for the experiment, a spectral bias is introduced in these averages; that is, the expected amplitude spectrum that would be obtained by randomly sampling noise tiles would have a shape close to 1/ f. We therefore examined differences between the averages in our categories and the expected bias. We obtained the bias by averaging the amplitude spectra of all the 4,900 noise tiles used to generate our stimuli. We then subtracted the bias from the averages obtained in each category to form what we will refer to as average difference spectra ( Figures 1f and 1g); this process is similar to the amplitude spectrum correction method described by Willmore and Smyth (2003). These average difference spectra represent dominant (relative to the bias) spatial frequency (indicated by the distance from the origin, ω, see Figure 1f) and orientation (indicated by the angle, θ, from vertical orientation, 0°, see Figure 1f) of the noise tiles within each category. Additionally, we zeroed the DC and 1 cycle/deg components, then smoothed each image with a 3 × 3 pixel Gaussian mask with σ = 0.9 pixel to improve visualization. Setting the very low frequencies (DC and 1 cycle/deg) to zero simply allows the full color map to be used for the more interesting spectral structures in surrounding frequency components (we have added a figure in the Supplementary materials section showing examples in which both components were left intact; the bandpass structure emphasized later in the paper is still obviously present). Note that no significant patterns were obtained by directly averaging, pixel by pixel, in the spatial domain (i.e., retaining the phase information). 

Results

Figure 2 shows the average difference spectra for the two false alarm categories obtained for the first set of 700 trials (first column), the second set of 700 trials collected approximately 1 month later (second column), and all 1,400 trials (third column) for each observer. For each observer and each set of trials, amplitude spectra were created using about 210 and 2,800 noise tiles, respectively, for the foveal and nonfoveal categories. Regions in red and blue indicate frequency components having amplitudes above and below the spectral bias, respectively (i.e., above and below the expected amplitude spectrum for randomly selected tiles). Regions in green show frequency components close to the bias. Surprisingly, each observer shows an idiosyncratic preference for certain distinct orientations. Further, note the high degree of similarity within observers between each set of 700 trials, particularly in the

$\bar{f}$

_FA category, which indicates the stability over time of these somewhat curious results. We have quantified these similarities, using zero-lag correlation between the smoothed average difference spectra of the two sets, for each observer, and we have obtained on average 0.72 (ranging from 0.6 for A.J.S. to 0.8 for A.T.). In the fourth column of Figure 2, we have cropped and enlarged the results from the third column to better visualize the spectral structures, and we have indicated the spatial frequency of the sought target (for the horizontal and vertical orientations). Notice that the peaks are close to the spatial frequency of the sought target (8 cycles/deg). 

Figure 2

 

Average difference spectra and performance plots for four observers. Average difference spectra, smoothed and contrast-stretched for visual enhancement, are shown for the first set of 700 trials (first column), the second set of 700 trials collected approximately 1 month later (second column), and for all the 1,400 trials (third column). For each observer and each set of trials, the spectra were created using about 210 and 2,800 noise tiles, respectively, for the foveal and nonfoveal categories. Regions in red and blue indicate frequency components having amplitudes above and below the spectral bias, respectively. Regions in green show frequency components close to the bias. In the fourth column, we have cropped and enlarged the results from the third column to better visualize the spectral structures, and we have indicated the spatial frequency (8 cycles/deg) of the search target (for the horizontal and vertical orientations). Observers performance (correct target detection rate, on a scale 0 to 1) are shown as a function of the orientation of the Gabor patches (pooled into 15° bins and averaged, each bin containing about 117 trials). We indicate in red the average performance of each observer across all orientations (this is close to the 68%). Regions in yellow and gray indicate performance above and below the observer's average performance, respectively.

We did not notice any reliable, dramatic effects of saccade length (which might be expected due to the falloff of resolution of the visual system). Nevertheless, an analysis of the average difference spectra binned by eccentricity (i.e., saccade length) for the nonfoveal categories is shown in the Supplementary materials section. 

We show in the last column of Figure 2 observers' performance in finding the Gabor target as a function of its orientation. Performance was pooled into 15° bins (12 bins total) and then averaged. Each bin contains about 117 trials. We indicate in red the average performance of each observer across all orientations (which is close to the initial value of 68% correct sought using the QUEST procedure). Regions in yellow and gray indicate performance above and below the observer's average performance, respectively. Notice that peaks in the average difference spectra for the f _FA category correspond quite closely to increases in performance at similar orientations. A repeated measures analysis of variance (ANOVA) with orientation and observer as factors showed a significant effect of orientation, F(11,705) = 5.46, p = 2.17 × 10 ⁻⁸, on the performance in finding the target, but a marginal effect of observer (note that the QUEST procedure described in the Methods section ensured similar average performance for observers). 

We have also tested whether behavior in a given trial is affected by the outcome of the preceding trial. Essentially, we wished to establish if the orientation of the target in trial τ _n−1 affected the outcome of trial τ _n; that is, if a delusive sequential strategy or bias permeates the observer's results (observers were told that the target orientation was chosen at random for each trial). We introduce the rotated average difference spectra obtained by rotating the noise tiles at trial τ _n by the negative of the orientation of the target at trial τ _n−1, then averaging them across trials. For example, if the target at trial τ ₅₄ has an orientation of 75°, we would rotate all the noise tiles in trial τ ₅₅ by −75° before averaging them in the FA categories. This process is designed to highlight dependencies between successive trials. For instance, if on average the observer tends to look for a similar orientation as the target in the preceding trial, then we would expect to see strong increases in amplitude close to the reference orientation ( θ _Ref), set at 0°. If there are no dependencies then we should observe an annulus (denoting an isotropic distribution) of the orientations. Figure 3 shows that for all four observers the

$\bar{f}$

_FA categories present structures close to an annulus, therefore that there appears to be, on average, no significant sequential bias. For observers A.T. and A.E.P., the f _FA category shows some biases; for example, a wide spread of high amplitude frequency components orthogonal to the reference orientation is observed for A.T., possibly suggesting a decrease in frequency detection accuracy for orientations perpendicular to the previous trial, or a slight tendency to select perpendicular orientations from trial to trial. 

Figure 3

 

Test for sequential bias. Rotated average difference spectra, smoothed and contrast stretched for visual enhancement, are shown. Regions in red and blue indicate frequency components having amplitudes above and below the spectral bias, respectively. Regions in green show frequency components close to the bias. See text for details.

Discussion

The main objective of this study was to investigate saccadic targeting and target selection in a naturalistic visual search task, when observers sought a randomly oriented Gabor target in a grid of 1/ f noise (which has an amplitude spectrum distinctive to natural scenes). We are interested in discovering what attracts fixations and how target candidates are selected upon fixation, in particular, when observers are uncertain about a target feature (here, the orientation). 

The results clearly point to visual guidance in saccadic target selection (in particular, under orientation uncertainty), a much debated issue in previous studies of active visual search (Findlay, 1997; Findlay & Gilchrist, 2003; Hooge & Erkelens, 1999; Motter & Belky, 1998; Zelinsky, 1996). We demonstrate that observers are selective for spatial frequencies close to the central frequency of the sought target; that is, the average difference spectra for fixated noise tiles show peaks localized in spatial frequency (close to 8 cycles/deg) but spread across various orientations (see Figure 2). Note that, in earlier work (Tavassoli, van der Linde, Bovik, & Cormack, 2006), we found that observers were selective for both spatial frequency and orientation when the orientation of the target was known (we used Gabor targets of spatial frequency 8 cycles/deg and fixed orientation {0°, 20°, 45°, 70°, 90°} in five separate experiments). Figure 4 shows examples of results obtained for observer I.V.D.L. (similar results were obtained for two other observers). 

Figure 4

 

Examples from previous findings. Average difference spectra are shown for observer I. V. D. L. using Gabor targets of spatial frequency 8 cycles/deg and fixed orientation {0°, 20°, 45°, 70°, 90°} in five separate experiments of 700 trials each.

Surprisingly, even under conditions of complete orientation uncertainty, observers show pronounced, idiosyncratic biases for certain stimulus orientations in saccadic programming; that is, rotational smearing of the peaks in the average difference spectra for the fixated noise tiles is limited to a subset of orientations. Note that if observers were equally selective for all stimulus orientations, one would expect to obtain peaks spread across all orientations, hence giving rise to a full annulus structure in the Fourier domain. Interestingly, these preferences are not exclusively limited to the cardinal directions (except for observer I.V.D.L.), which is somewhat inconsistent with many physiological and behavioral studies (Berkley et al., 1975; Campbell et al., 1966; Li et al., 2003), although the strongest preference across all observers appears to be close to vertical (0°). Nor are the biases solely reserved to the oblique orientations, as might be expected given the results of behavioral studies using more naturalistic stimuli (Hansen & Essock, 2004). Instead, we demonstrate preferences for a subset of orientations that encompass, in part, cardinal and oblique orientations. 

These biases are also present in observers' performance data, and, within each observer, they are remarkably similar to the biases seen in average difference spectra for noise tiles selected as target candidates upon fixation (although more trials would be needed to examine the fine structure, if any, of this similarity). We show that the performance in finding the target is dependent on its orientation and that an asymmetry exists between clockwise and anticlockwise orientations; A.J.S., A.T., and I.V.D.L. have a preference for anticlockwise oriented stimuli whereas A.E.P. has a bias toward clockwise. Although mysterious in origin, such asymmetries have been reported in physiological studies of macaque monkeys (Finlay et al., 1976) and appear in earlier behavioral data (Boltz, Harwerth, & Smith, 1979). This observation may be a consequence of unequal populations or firing strength of orientation-tuned cells involved in the task, or may result from the assembly of search filters tuned such that certain orientations are amplified in sensitivity at the expense of others, in a dynamically reconfigurable, task-dependent manner. One may also speculate that these orientation biases could be related to observers' daily interactions with their unique environments, although further investigation would be required to substantiate these possibilities. 

Conclusions

Our results offer insight into observer behavior in visual search tasks under uncertain stimulus conditions. In our experiment, the spatial frequency was held constant while the orientation varied. We found that the observers relied on an invariant target feature, namely, spatial frequency structure similar to the sought target. Surprisingly, despite having no previous knowledge of each target's orientation, observers showed clear idiosyncratic biases in orientation selectivity during saccadic programming. These biases were also present in observers' foveal detection data and showed asymmetries between clockwise and anticlockwise orientations. Further examination of the effects of learning (e.g., training to least preferred orientations) may be useful in understanding mechanisms of plasticity in such tasks and is currently being studied. 

Supplementary Materials

Supplementary Figure - Supplementary Figure 

Figure SM1. Average difference spectra, smoothed and contrast-stretched for visual enhancement, are shown for observers AJS and AEP for all 1,400 trails, (A) without zeroing the DC and 1 c/deg components and (B) when the zeroing of the components is performed. Note that in (B) the positive peaks near the target frequency are somewhat enhanced and, moreover, clear negative regions are revealed. 

Supplementary Figure - Supplementary Figure 

Figure SM2. Average difference spectra for the non-foveal false alarm category binned by eccentricity (columns) for each observer (rows). The first four bins are 1 deg wide and centered at 1, 2, 3 and 4 deg, and the fifth encompasses everything above 4.5 deg (because saccades that large were relatively rare). The spectra have been smoothed and contrast-stretched to emphasize the spatial structure. Regions in red and blue indicate amplitudes above and below the overall average noise spectrum (~ 1/f), respectively, whereas regions in green show frequency components close the overall average (indicating frequency components playing no role on attracting the observers' gaze). Note that, if so desired, the signal-to-noise ratio for the spectra from larger saccades could be further improved by filtering them with respect to the decrease in the resolution of the visual system as a function of eccentricity. 

Acknowledgments

We thank Jeremy M. Wolfe for his valuable suggestions for this work. We would also like to acknowledge Wilson S. Geisler and Eyal Seidemann for their useful comments. This research was funded by NSF grants ECS-022545 and ITR-0427372. 

Commercial relationships: none. 

Corresponding author: Abtine Tavassoli. 

Email: [e-mail]. 

Address: Center for Perceptual Systems, 1 University Station, A8000, University of Texas, Austin, TX 78712, USA.