Out of the corner of the driver's eye: Peripheral processing of hazards in static traffic scenes

Abstract

Effective gaze control in traffic, based on peripheral visual information, is important to avoid hazards. Whereas previous hazard perception research mainly focused on skill-component development (e.g., orientation and hazard processing), little is known about the role and dynamics of peripheral vision in hazard perception. We analyzed eye movement data from a study in which participants scanned static traffic scenes including medium-level versus dangerous hazards and focused on characteristics of fixations prior to entering the hazard region. We found that initial saccade amplitudes into the hazard region were substantially longer for dangerous (vs. medium-level) hazards, irrespective of participants' driving expertise. An analysis of the temporal dynamics of this hazard-level dependent saccade targeting distance effect revealed that peripheral hazard-level processing occurred around 200–400 ms during the course of the fixation prior to entering the hazard region. An additional psychophysical hazard detection experiment, in which hazard eccentricity was manipulated, revealed better detection for dangerous (vs. medium-level) hazards in both central and peripheral vision. Furthermore, we observed a significant perceptual decline from center to periphery for medium (but not for highly) dangerous hazards. Overall, the results suggest that hazard processing is remarkably effective in peripheral vision and utilized to guide the eyes toward potential hazards.

Introduction

Efficient interaction of visual processing and oculomotor control is essential for safely navigating in traffic. One important aspect of safe navigation is the ability to detect and process potential hazards in order to avoid accidents. Accident research indicated that visual orientation problems are indeed a major cause for many accidents (e.g., Dingus et al., 2006; Horswill & McKenna, 2004; Pelz & Krupat, 1974). The present research aims at shedding more light on peripheral hazard detection during visual orientation in traffic scenes. 

It is well known from research on oculomotor control in scene perception that visual scanning of a scene does not proceed in a random fashion (Rayner, 2009). Instead, it is assumed that each fixation not only involves processing of information at the fovea, but also in the periphery to guide the eyes towards relevant information in the upcoming fixation. The fovea typically comprises 1° of eccentricity, whereas the parafovea extends from 1° to about 4°–5°. Together, fovea and parafovea are commonly referred to as central vision. In contrast, peripheral vision, which is known to be substantially degraded as compared to central vision, encompasses the remainder of the visual field (e.g., Larson & Loschky, 2009). Despite its low resolution, peripheral processing is a necessary prerequisite for efficient eye guidance in scenes, and is subject to both bottom-up and top-down influence. Whereas such bottom-up processing is conceptualized as being based on physical saliency (i.e., objects or regions with greater color or luminance contrast are preferentially processed as candidates for the next fixation; see Itti, & Koch, 2000, 2001; Peters, Iyer, Itti, & Koch, 2005), top-down processing (e.g., Bindemann, 2010) can involve either mandatory processing (based on experience, e.g., schema effects) or volitional processing (based on specific task instructions or an act of will; see Baluch & Itti, 2011). In real-life scenarios, top-down sources of influence can be very strong up to a point where they overrule the influence of bottom-up factors (e.g., Foulsham, Alan, & Kingstone, 2011; Henderson, Brockmole, Castelhano, & Mack, 2007). 

A crucial issue in research on peripheral processing is the question of how far in peripheral vision objects can be processed, a distance that will depend on many factors such as task demands, visual resolution at each eccentricity, object size, object contrast, the presence of nearby flanking objects, relevant object features, and saliency. For example, on the one hand (foveally or generally) demanding tasks usually go hand in hand with a narrowing of peripheral processing efficiency (e.g., Henderson, Williams, Castelhano, & Falk, 2003, who utilized a memory-based task in which object deletion/substitutions in a scene had to be detected; see also Ringer, Throneburg, Johnson, Kramer, & Loschky, 2016), whereas on the other hand, for example, judgments related to animal presence in natural scenes (i.e., an assumedly easier task) are associated with impressive perceptual abilities in the far periphery (Thorpe, Gegenfurtner, Fabre-Thorpe, & Bulthoff, 2001). Based on this evidence for a major role of top-down factors involved in spatial saccade target selection, we hypothesized that the visual system should also be highly efficient in processing visual information from the periphery with respect to its hazard proneness, since hazards should be inherently attention-grabbing (especially when task demands and instructions highlight the processing of potential hazards). Whereas several previous studies already addressed the issue of peripheral processing in traffic on a more general level (e.g., Bronstad, Bowers, Albu, Goldstein, & Peli, 2013; Clay et al., 2005; Crundall, Underwood, & Chapman, 1999; Horrey, Wickens, & Consalus, 2006; Lamble, Laakso, & Summala, 1999; McCarley et al., 2004; Pringle, Kramer, & Irwin, 2004; Sanocki, Islam, Doyon, & Lee, 2015; Shinoda, Hayhoe, & Shrivastava, 2001; Summala, Lamble, & Laakso, 1998), the present study is specifically focused on the underlying spatio-temporal dynamics and the role of hazard eccentricity. 

One interesting aspect of this hypothesized ability to process hazards in the periphery is that unlike in typical visual search tasks used in basic research involving the exact presentation of a target object (e.g., a tilted, colored line) prior to a search display (e.g., Treisman & Gelade, 1980), the viewer of a traffic scene only knows to a certain degree where to look and what to look for (e.g., red traffic lights, cars coming from the right), and the potential targets are more numerous (i.e., the memory set is larger) and rather vaguely defined (e.g., you do not know what sort of potential hazard may enter the road way, such as a child, a bicyclist, or another car, etc.). Thus, any ability to process the degree of hazard potential in peripheral vision should represent a fairly complex and abstract accomplishment of the visual processing system, since it cannot solely rely on, for example, preactivation of basic features like color-, orientation- and form-related information, as conceptualized in theories of guided search (e.g., Wolfe, 1994). 

In a previous study on searching for potential hazards in static traffic scenes (Huestegge, Skottke, Anders, Debus, & Müsseler, 2010), we built on other research suggesting that hazard perception (in terms of overall response time, RT) is faster for experienced versus novice drivers (e.g., Grayson & Sexton, 2002; Hull & Christie, 1993). Specifically, we compared eye movements of 20 inexperienced and 20 experienced drivers in a hazard perception task involving static traffic scenes. A key feature of this previous study was that we did not only analyze overall RT (i.e., the time between scene onset and response initiation). Instead, we separately measured (a) the interval between the onset of a static hazard scene and the first fixation on a potential hazard (i.e., visual orientation latency), and (b) the interval between the first (potential) hazard fixation and the final response (i.e., hazard processing duration). Whereas overall RT was faster for experienced versus inexperienced drivers, the scanning patterns revealed that this advantage was due to faster hazard processing, whereas visual orientation latency was comparable between groups. However, in this previous study we did not address any processes underlying hazard perception in peripheral vision. 

Here, we reanalyzed the data of this previous study in order to assess the extent to which hazard level (i.e., whether a potential hazard implies medium vs. high braking affordance) can be processed in peripheral vision. Additionally, we were interested in learning more about the underlying temporal processing dynamics. Whereas these novel research questions cannot be answered on the basis of our previous analyses, our previous data set was considered to be perfectly suited to empirically address this issue. As such, the present reanalysis represents a case of rigorous hypothesis testing (as opposed to posthoc rationalization) in that the hypotheses were derived prior to (and independent from) a corresponding look into the data. 

To shed light on these issues, our reanalysis focused on spatial and temporal parameters of oculomotor control immediately prior to entering the hazard region. Specifically, we hypothesized that (a) if hazard-level processing is possible based on information from peripheral vision, we would expect that highly dangerous hazards are targeted from a further distance than medium dangerous hazards. We will refer to this as the hazard-level dependent saccade targeting distance effect. This hypothesis has been derived from similar observations in other research fields. For example, searching for a face is known to be facilitated for faces with direct gaze towards the participant (vs. averted gaze; Von Grünau & Anston, 1995) regardless of the task relevance of gaze direction information (e.g., Doi & Shinohara, 2013), presumably because faces exhibiting direct gaze are highly relevant for an observer on a social level. In the same way, high-level hazards can also be considered personally relevant, which may imply a top-down generated search priority for this particular object category (vs. medium-level hazards) in peripheral vision. Of course, a prerequisite for the effectiveness of this processing priority is the ability to identify the associated object characteristics in the periphery. Furthermore, (b) if these computations are really a product of online processing during the fixation prior to the first hazard fixation, we would expect to find a dependency of this effect on the duration of the previous fixation: That is, fixations that are too short to allow for peripheral information processing should not be associated with a hazard-level dependent saccade targeting distance effect. Specifically, we assume that in complex naturalistic traffic situations, processing load both at foveal and peripheral locations should be comparatively high, so that a certain amount of fixation time is needed to successfully compute a rather abstract feature such as the hazard proneness of a visual configuration in the periphery. Thus, the second hypothesis also represents further assurance against a random positive finding for the first hypothesis. Finally, we also included the group variable “expertise” in our analyses to explore to what extent peripheral processing efficiency might depend on driving expertise. In addition to this reanalysis, we conducted a new psychophysical hazard detection experiment (without eye movement recording but based on the same scene material) in which hazard eccentricity was systematically manipulated to study hazard detection as a function of hazard eccentricity (central vs. peripheral) under more controlled conditions. 

Reanalysis of eye movements in static traffic scenes involving hazards

Method

The following is a somewhat condensed version of the methods used in Huestegge et al. (2010), and for more details, the interested reader can consult that study. 

Participants

We reanalyzed eye movements of 40 drivers which were divided into two equally sized groups of experienced versus inexperienced drivers (see Huestegge et al., 2010, for more details). All had normal or corrected-to-normal vision. 

Materials

Stimuli consisted of 89 real-life photographs depicting traffic situations from the drivers' perspective. The pictures contained situations of variable braking affordance, such as road works, flashed braking lights of a car ahead, pedestrians, or children. Three experts (driving instructors) independently categorized the pictures into three sets of pictures with low, medium, and high braking affordance (see Figure 1 for examples). The assignment of the scenes to the three categories was validated in a previous study (Biermann, Skottke, Brünken, Debus, & Leutner, 2008), which showed that the scene categories differed regarding mean RTs (with respect to the medium and high affordance category) and the number of initiated braking responses (all three categories). In the study relevant for the present reanalysis, we utilized a subset of the Biermann et al. (2008) database, namely 26 high-level, 39 medium-level, and 24 low-level hazard scenes. 

Figure 1

 

Examples for traffic scenes (from left to right: low, medium, and high braking affordance) used in the study. Rectangles (invisible for participants) indicate the potential hazard region.

The low braking affordance scenes only served as fillers in the study, since there was no clear region in these scenes indicating a potential hazard. In the pictures depicting medium and high braking affordance, rectangular regions (including a fixed spatial margin to account for spatial eye tracking inaccuracies) were drawn around the potential hazard (see Figure 1). It was ensured that the overall mean size of these hazard regions as well as their spatial distance from the screen center and from the fixation cross that occurred at the beginning of each trial (see below) did not differ between both categories. Specifically, there were no statistically significant differences in the height (high-level hazards: M = 4.14, SD = 1.04, medium-level hazards: M = 4.04, SD = 1.25) and width (high-level hazards: M = 4.86, SD = 1.38, medium-level hazards: M = 4.55, SD = 1.02) of the hazard regions between the two hazard categories, both ts < 1. There were also no significant differences in the spatial distance of hazards from the screen center (high-level hazards: M = 2.26, SD = 1.45, medium-level hazards: M = 2.54, SD = 1.82), t < 1, and from the upper left fixation cross (high-level hazards: M = 11.82, SD = 2.17, medium-level hazards: M = 12.74, SD = 2.20), t < 1.3. In the context of the previous study (Huestegge et al., 2010), we also controlled for the saliency of hazard regions across the two scene types (medium vs. high braking affordance) based on the Itti and Koch (2000) algorithms to ensure that any differences in processing between the two scene types cannot be explained in terms of saliency differences. Specifically, we used the “ezvision” routine of the “Saliency” software packet (see Itti & Koch, 2000) using standard parameters (i.e., incorporating color, intensity, and orientation characteristics). We computed whether at least one of the ten most salient locations in each traffic scene (as computed by ezvision) fell into the hazard region and categorized these hazards as salient, whereas hazards which did not spatially correspond to one of the ten most salient locations were considered as being of low saliency. This dichotomous categorization procedure resulted in a roughly equal distribution of salient (44%) versus nonsalient (56 %) hazard scenes. Specifically, the relative distribution of salient versus nonsalient regions did not significantly differ between medium-level (salient: 50%, nonsalient: 50%) and highly dangerous (salient: 39%, nonsalient: 61%) hazard categories (p > 0.29 based on Fisher's exact test). As an additional continuous visual saliency measure, we computed relative hazard saliency by dividing the saliency of the hazard region by the saliency of the complete picture (again based on the algorithm by Itti & Koch, 2000). As in the previous saliency analysis, relative hazard saliency did not differ between both categories (high-level hazards: M = 1.09, SD = 0.31, medium-level hazards: M = 1.13, SD = 0.30), t < 1. 

Procedure

Participants were seated at a distance of 70 cm in front of a 21″ CRT monitor. A chinrest was used to minimize head movements. Eye movements were recorded using a head mounted EyeLink I system (SR Research, Osgoode, Canada) with a temporal resolution of 250 Hz. Eighty-nine pictures (subtending a visual angle of 24.0° × 18.6°) depicting the three scene types were shown for two seconds each in a fixed, prerandomized sequence. They were separated by a black screen for 1000 ms, followed by a white fixation cross in the upper left corner of the black screen to ensure a controlled starting position for visual scanning of the subsequent picture. Participants were instructed to respond as quickly as possible to those scenes which subjectively demanded a braking response or speed reduction by pushing the space button of a keyboard in front of them (serving as a proxy for braking responses). Recalibrations of the eye tracker were conducted after every five trials. 

Design

Relevant dependent variables in the reanalysis study (e.g., mean incoming saccade amplitudes for the hazard regions) were analyzed as a function of hazard level (medium vs. high braking affordance, within-subject factor) and expertise (inexperienced vs. experienced drivers, between-subjects factor) using a 2 × 2 mixed ANOVA (unless otherwise indicated). We then analyzed the dependency of the hazard-level effect on the duration of the previous fixation to gain further insight into the underlying processing dynamics. In the case of significant ANOVA results, effect sizes (η_p²) are reported. Other general results (see also Huestegge et al., 2010) relevant to the present reanalysis will be briefly summarized. 

Results and discussion

General results

Thirteen percent of the trials from the experienced drivers and 15% of the trials from the inexperienced drivers were removed from the analysis due to impaired eye movement data (including trials with eye movements registered outside the scene) or RTs deviating ±3 SDs. As already reported in Huestegge et al. (2010), the probability of initiating a key press response was higher for high braking affordance scenes (M = 89.6%) as compared to both medium braking affordance scenes (M = 45.4%) and low braking affordance scenes (M = 15.1%), without any impact of expertise. In trials where a response was initiated (in the medium and high hazard-level scenes), overall RTs (interval between picture onset and response) were faster for experienced versus inexperienced drivers and for high- versus medium-level hazard scenes, confirming the assumed difference in braking affordance between hazard levels. As mentioned in the Introduction, the decomposition analysis of RTs revealed that the expertise advantage was due to faster hazard processing time, not visual orientation time (see Huestegge et al., 2010). However, there were also effects of hazard level: Hazards with high (vs. medium) braking affordance were fixated earlier and processed faster, whereas there were no interactions of expertise and hazard level. An analysis of mean fixation durations in Huestegge et al. (2010) only considered durations of > 70 ms and < 1000 ms. These mean fixation durations were not affected by any of the independent variables. However, high hazard-level scenes were inspected with significantly fewer fixations (M = 3.05) as compared to medium hazard-level scenes (M = 3.27), reflecting the difference in overall RTs. Note that the first fixation in the upper left corner (triggered by the preceding fixation cross) was not included in this analysis. Especially relevant with respect to the analyses of the initial saccade into the hazard region below, we observed that high (vs. medium) hazard-level scenes were scanned with significantly longer overall saccade amplitudes. This observation is difficult to interpret without reference to the specific goals associated with each saccade, and thus clearly calls for a more in-depth analysis especially regarding the amplitude of the initial saccade into the hazard region as a function of hazard level. 

In the context of the present reanalysis, we further analyzed all trials that received a “hazard present” response by participants in order to assess how many of the high-level and medium-level hazard regions (that were classified as hazards) actually received at least one fixation. Interestingly, whereas 96.2% (SD = 0.22) of the high-level hazards received at least one fixation, only 40.3% (SD = 0.35) of the medium-level hazards received a fixation prior to being classified as “present”, F(1, 38) = 952.47, p < 0 .001, η_p² = 0.992. Obviously, many medium-level hazards were classified as being present based on extrafoveal vision. A similar picture emerged when we compared the mean number of fixations on the (medium-level vs. high-level) hazard region for those cases in which at least one fixation on the hazard region occurred. Here, we found that high-level hazard regions received a mean of 2.33 fixations (SD = 0.31) versus 1.83 fixations (SD = 0.35) for medium-level hazards, F(1, 38) = 49.02, p < 0.001, η_p² = 0.56. Note that the possibility that medium-level (vs. high-level) hazards were associated with a shift in response bias will be tested more explicitly in the psychophysical experiment, and further potential explanations for the reported differences will be outlined in the General discussion (see below). There were no significant main effects of (or interactions with) expertise, all ps > 0.10. 

Hazard-level dependent saccade targeting distance effect

For this central analysis, we computed the incoming saccade amplitude for each hazard region (only for the first saccade into the hazard region in each trial) whenever the latter received at least one fixation prior to responding. Only trials without the occurrence of blinks (amounting to 3% of all trials) were considered. We then submitted these data to a mixed 2 (hazard level) × 2 (expertise group) ANOVA. As a result (see Figure 2), we found a significant main effect of hazard level, F(1, 38) = 19.33, p < 0.001, η_p² = 0.337, indicating that the mean initial incoming saccade amplitude into the hazard region was greater (M = 6.85°, SD = 1.13) for hazards with high braking affordance (high hazard level) than for hazard with medium braking affordance (medium hazard level) (M = 5.54°, SD = 1.05). There was no significant main effect of expertise, F(1, 38) = 1.87, p = 0.18, and no significant two-way interaction, F < 1. Taken together, these results clearly indicate the presence of a hazard-level dependent saccade targeting distance effect in our data. To rule out that this effect is due to an attentional bias towards persons depicted in the hazard region (five high-level scenes contained the selective presence of a person in the hazard region, whereas this was only the case in one medium-level scene), we reran the analysis excluding the five scenes selectively involving a person in the high-level hazard condition. As a result, the main effect of hazard level on incoming saccade amplitude remained unaltered (high hazard level: M = 7.09°, SD = 1.21, medium hazard level: M = 5.55°, SD = 1.05), F(1, 38) = 25.73, p < 0.001, η_p² = 0.404. The hazard-level dependent effect on incoming saccade amplitudes also accounted for the finding that the overall mean saccade amplitude during scene scanning was longer for high-level (vs. medium-level) hazard scenes, since an analysis of mean saccade amplitudes during scene scanning excluding these initial saccades into the hazard region no longer yielded a statistically significant main effect of hazard level category, F < 1. 

Figure 2

 

Initial saccade amplitude towards hazard region as a function of hazard level and driving expertise. Error bars represent standard errors.

Despite the nonsignificant difference of the number of salient versus nonsalient hazard regions across the two hazard categories (see Method section), we reran the analysis described above but additionally included saliency (relative hazard saliency, see methods section) as a within-subject factor in the ANOVA. This analysis revealed neither a significant main effect of saliency, nor a significant interaction of saliency with any of the other factors (hazard level and expertise), all Fs < 1. Thus, we can rule out the possibility that differences in saliency significantly contributed to the observed effect. 

One alternative explanation of the hazard-level dependent saccade targeting distance effect could be that previous fixations (i.e., fixations that occurred prior to the last fixation outside the hazard region) were already located close to the hazards especially for the medium-level hazard scenes, thus allowing for hazard preprocessing. To address this issue, we further analyzed the Euclidean distance of these early fixations from the hazard region. For this analysis, the label Fixation N-1 refers to the fixation immediately prior to entering the hazard region, while the preceding fixations are labelled Fixation N-2 and Fixation N-3, respectively. The results clearly rule out the possibility that Fixation N-2 and N-3 (if present at all in a trial) were located nearby the hazard region in both hazard-level conditions: ANOVAs with the within-subject factor fixation N (number) (comparing the hazard distance of Fixation N-2 vs. N-1 and N-3 vs. N-1) and the between-subject factor expertise revealed that the mean location of Fixation N-2 was significantly farther away from the hazard region when compared to the location of Fixation N-1, F(1, 38) = 250.85, p < 0.001, η_p² = 0.87 for high-level hazards and F(1, 38) = 194.11, p < 0.001, η_p² = 0.84 for medium-level hazards, respectively. The same held for Fixation N-3, which was also further away from the hazard as compared with Fixation N-1, F(1, 38) = 680.92, p < 0.001, η_p² = 0.94 for high-level hazards and F(1, 38) = 99.70, p < 0.001, η_p² = 0.72 for medium-level hazards, respectively. There was no significant main effect of (or interaction with) expertise, all ps > 0.10. Thus, the hazard-level dependent saccade targeting distance effect cannot be explained in terms of spatial fixation history. 

Dependency of the targeting distance effect on the time elapsed during Fixation N-1

To further analyze the underlying processing dynamics of hazard level during the fixation immediately prior to the initial hazard fixation (Fixation N-1), we computed the mean initial saccade amplitude into hazards with medium versus high braking affordance as a function of the time that had elapsed for Fixation N-1. Specifically, we ran the same statistical analysis as described in the previous subsection (first analysis), but prior to averaging the single-trial data for each participant we selected only a certain range of Fixations N-1 based on their duration. For example, we selected only trials that included Fixations N-1 of < 130 ms, < 160 ms, < 190 ms, etc. (see Figure 3). Note that this procedure implies that each step involves the contribution of additional trials, which is also reflected in decreasing SEs in Figure 3 from left to right. Note that SEs even from the first data point (which involves only relatively few trials) appears to be small enough to allow for a meaningful interpretation. The final data point in Figure 3 involves all trials and is thus redundant with the data in Figure 2 (averaged across expertise conditions). Note that we did not analyze statistically independent bins of fixation durations (e.g., 130–160 ms, 160–190 ms, etc.) since such an analysis would be severely statistically underpowered (and/or substantially less fine-grained). 

Figure 3

 

Development of the hazard-level dependent targeting distance effect as a function of an increasing selection criterion for the fixation duration prior to initial hazard fixation. Error bars represent SE. Note that the data points from left to right are not statistically independent (i.e., the latter contain the data from the former).

Whereas there were no significant hazard-level dependent saccade targeting distance effects for N-1 fixation durations below 220 ms, a statistically significant effect emerged at around 220 ms and gradually increased until about 400 ms, where it has reached its full potential. It is interesting to note that the sample of short fixation durations in the left part of Figure 3 is associated with relatively longer saccade amplitudes into the hazard region, suggesting that short fixation durations which did not allow for in-depth hazard-level processing in the periphery were associated with relatively long saccade amplitudes (see Unema, Pannasch, Joos, & Velichkovsky, 2005; Velichkovsky et al., 2003). 

Psychophysical experiment

Method

Participants

The psychophysical experiment included 42 new participants (32 female, 10 male; mean age = 26 years, SD = 7). All had normal or corrected-to-normal vision. 

Materials

A selection of the scenes from the reanalysis study referred to above were employed in the psychophysical experiment. We only utilized scenes with clearly defined, unitary hazard regions and aimed at equal distribution of scenes suggesting braking/no braking, resulting in 24 low-level, 12 medium-level, and 12 highly dangerous hazard scenes. Height (high-level hazards: M = 4.52, SD = 1.24, medium-level hazards: M = 4.14, SD = 1.31) and width (high-level hazards: M = 4.84, SD = 1.13, medium-level hazards: M = 4.67, SD = 1.05) of the hazard regions as well as their visual saliency (in terms of relative hazard saliency, high-level hazards: M = 1.09, SD = 0.32, medium-level hazards: M = 1.04, SD = 0.26) did not differ significantly between medium-level versus highly dangerous hazards, all ts < 1. 

Procedure

Participants were seated in front of a 17″ CRT monitor with a viewing distance of 70 cm. Picture size (absolute and in degree visual angle) was the same as in the reanalysis study (see above). Each trial started with the presentation of a white central fixation cross on black background. Subsequently, the scene was briefly shown for 250 ms, followed by a black screen (until participants' response, maximum 3000 ms). Intertrial intervals varied randomly between 900 and 1100 ms. Participants were asked to press the space bar as quickly as possible when they would initiate a braking response based on the presence of a hazard (similar to a go/no-go task) and to return their gaze to the center of the screen at the end of each trial. The experiment was divided into two consecutive blocks; each block consisted of the same 48 trials displayed in randomized order. The administration of two similar, consecutive blocks was introduced to achieve more reliable performance estimates. To manipulate eccentricity, the scenes were displayed in a way that the center of the hazard region was located at the screen center (0° visual angle) or in the periphery (5.5° or 7° visual angle, based on the findings from the eye movement reanalysis, see above). Importantly, each stimulus was only presented at one visual angle condition for each participant. The assignment of stimuli to visual angle conditions was counterbalanced across participants, and it was ensured that every stimulus was presented equally often at all three eccentricity conditions. Whereas there were no definable eccentricity conditions for the stimuli displaying low-level hazard scenes due to the lack of a definable hazard region, these scenes were also presented at varying locations (i.e., similar to the stimuli depicting medium- or high-level hazard scenes). 

Design

We analyzed the percentage of key press responses and response times (RTs) for the different hazard-level conditions as a function of eccentricity. In the case of significant ANOVA results, effect sizes (η_p²) are reported. 

Results and discussion

An overview of the results is depicted in Figure 4. The probability of initiating a key press response was 86.7% (SD = 13.4) for high braking affordance scenes, 58.2% (SD = 20.7) for medium braking affordance scenes, and 11.9% (SD = 10.0) for low braking affordance scenes, F(2, 88) = 445.3, p < 0.001, η_p² = 0.916. Posthoc contrasts revealed significant differences between all hazard-level categories, all ps < 0.001. Note that the probability values are roughly comparable to the previous eye-tracking study (see above). However, there were no significant effects of eccentricity, neither in the highly dangerous hazard category, nor in the medium-level hazard category (ps > 0.31 for all contrasts). From the perspective of signal detection theory (i.e., when we interpret responses in the high/medium hazard-level categories in terms of hits and responses in the low-level hazard condition as false alarms), these results clearly indicate a difference in sensitivity (detection accuracy) between hazard level categories. However, we refrained from explicitly calculating and statistically comparing sensitivity and response bias parameters due to the lack of statistically independent false alarm rates in the present research design. 

Figure 4

 

Results from the psychophysical experiment. The upper panel shows the mean key press probabilities as a function of hazard level and eccentricity. Note that whereas in the low-level hazard condition the position of the scenes on the screen varied in a similar way as in the other two conditions, there were no definable eccentricity parameters in the former (thus, only one bar is depicted). The lower panel depicts RTs as a function of eccentricity and hazard level. Error bars indicate standard errors.

RTs were only analyzed in the medium-level and highly dangerous hazard categories. RTs were significantly longer for medium-level (vs. highly dangerous) hazards (910 ms vs. 840 ms), F(1, 36) = 35.8, p < 0.001, η_p² = 0.499. Post hoc contrasts revealed significant effects of hazard level across all eccentricities (all ps < 0.01). Overall, RTs showed a nonsignificant trend to increase with eccentricity, F(2, 72) = 2.6, p = 0.081, η_p² = 0.068. Pairwise one-tailed contrasts revealed a significant RT increase in the periphery for medium-level hazards (contrast 0° vs. 5.5°: 56 ms, t(39) = 2.8, p = 0.004, contrast 0° vs. 7°: 45 ms, t(37) = 2.0, p = 0.027), whereas high-level hazards were detected rapidly without a significant RT increase in the periphery (all contrasts p > 0.16). However, the interaction of hazard level and eccentricity was not statistically significant, F(2, 72) = 1.1, p = 0.32. 

General discussion

The aim of the present study was to analyze the spatio-temporal characteristics and dynamics of hazard perception in peripheral vision by reanalyzing data from a previous eye movement study on hazard perception in traffic scenes (Huestegge et al., 2010) and by running a new psychophysical hazard detection experiment. In the eye movement study, participants scanned static traffic scenes including regions of varying hazard level (medium vs. dangerous) while their eye movements were monitored. In the present analysis, we specifically focused on characteristics (distance, duration) of the fixation immediately prior to entering the hazard region to assess both the extent and the dynamics of hazard processing in peripheral vision. In the psychophysical experiment, hazard eccentricity was manipulated to study hazard detection as a function of hazard eccentricity. 

As main results, we found that the amplitudes of initial saccades towards highly dangerous hazard regions were substantially longer than those towards medium-level hazard regions, a finding that could not be explained in terms of hazard saliency differences. This effect was present in a similar way for both inexperienced and experienced drivers. Additionally, an analysis of the temporal dynamics of this effect revealed that peripheral hazard-level processing occurred around 200–400 ms during the course of the fixation prior to entering the hazard region. This result is well in line with claims of previous research estimating that the time needed to encode the location of a target within the field of vision and to initiate an eye movement is of the order of 175–200 ms (e.g., Becker & Jürgens, 1979). Overall, the results suggest that visual information processing of hazard level in the periphery is remarkably effective and utilized to guide the eyes towards potential hazards. 

In the psychophysical experiment, we replicated several key findings from the eye movement study. Specifically, high-level hazards were more frequently and rapidly categorized as hazards than medium-level hazards. Furthermore, the detection rates in the medium-hazard category clearly differed from those in the low-level hazard category, indicating that participants were not just guessing regarding medium-level hazard presence but were indeed able to perceive medium-level hazards in the periphery. More specifically, high-level hazards could be seen with nearly perfect accuracy from at least 7° eccentricity, and medium level hazards with moderate accuracy from 5.5° eccentricity. 

However, some aspects of the eccentricity results were less clear. Descriptively, high-level hazards were found equally quickly across the whole range of eccentricities, whereas medium-level hazard detection speed decreased with increasing eccentricity. However, whereas performance in the periphery for high-level (vs. medium-level) hazards was clearly significantly faster, we did not find a statistically significant interaction, that is, statistical evidence for a stronger performance decrement towards the periphery for medium-level versus high-level hazards. This lack of a significant interaction might be due to relatively high standard deviations arising from the highly constrained item material, since only 12 scenes per hazard-level category were included in the experiment, and the assignment of scenes to eccentricity conditions needed to be counterbalanced across participants. Thus, it would be interesting to address this issue again in the future with a more extended range of item material and eccentricities. 

The data from the present set of eye movement analyses represents a good moment-to-moment indication of cognitive processes during traffic scene perception. During the course of visual scanning, attention is assumed to precede each saccade to a new spatial location within the scene (Deubel & Schneider, 1996; Henderson, 1992; Hoffman & Subramaniam, 1995; Kowler, Anderson, Dosher, & Blaser, 1995). Thus, the eyes move once foveal information has been processed, and a saccade target location has been chosen. By using a study design involving gaze-contingent masks of either central or peripheral vision at various points in time during fixations, van Diepen and d'Ydewalle (2003) suggested that the shift of attention from central vision to the periphery takes place quite rapidly during the course of a fixation, and attention is then directed to the periphery to choose a potential saccade target immediately following the processing of central information. However, the exact timing should greatly depend on the specific content of and fixation location within a scene, and in the case of the present hazard perception task, the ability to target a potential hazard in the periphery appears to occur at around 200 ms. 

Two observations from the eye movement study may at first sight appear to be contradictory. On the one hand, the initial saccade amplitudes into the hazard region suggest that highly dangerous hazards are easier to detect in the periphery when compared to medium-level hazards, and the data from the psychophysical experiment show that this was indeed the case. On the other hand, medium-level hazards received fewer direct fixations, suggesting that they were more likely to be indicated as being present based on peripheral vision, that is, without the need to be directly fixated. This would oddly seem to imply better peripheral detection for medium-level hazards. However, this explanation is not only inconsistent with the eye movement results for the high-level hazards, but is also directly refuted by the psychophysical experiment, which showed that the medium-level hazards were detected at a lower rate (and more slowly) than the high-level hazards. Thus, we can clearly reject such an explanation of the fixation probability in the medium-level hazard condition. Instead, it is conceivable that the observation of fewer direct fixations for medium-level hazards is due to the participants' instruction to initiate speeded responses. Whereas participants might perceive their response to highly dangerous hazards as sufficiently fast, they might feel stronger time pressure in the more time consuming medium-level hazard decision trials (as shown by the longer detection RTs for the medium level hazards in the psychophysical experiment). As a response to this time pressure, they might tend to save time by omitting a final direct hazard fixation prior to responding. 

To further understand the mechanisms underlying the hazard-level dependent saccade targeting distance effect, it is important to consider that the mean saccade amplitudes into the hazard region were greater than 5° for both hazard-level categories (i.e., corresponding to peripheral vision), suggesting that an abstract feature like hazard level can be estimated from more than 5° away during the previous fixation. At first sight, this may seem at odds with previous studies using moving window procedures (McConkie & Rayner, 1975) to estimate the size of the field of view around a fixation. These studies typically indicate that information uptake beyond 5° eccentricity is severely limited (e.g., Loschky & McConkie, 2002; Nuthmann, 2014; Shiori & Ikeda, 1989), and these limitations should become especially apparent for more abstract, higher level semantic features such as hazard level. However, Nuthmann (2013) showed that the visual span for object search in real-world scenes extended to at least 8° eccentricity, which is rather compatible with our results. 

Given that the present analysis involved realistic traffic scenes, it appears at least principally possible that the first gist of the scene along with real-world knowledge (e.g., memory schemata based on statistical learning) may have helped participants to make informed guesses where relevant hazard-related information might be located (see Henderson, 1992; Larson & Loschky, 2009; Potter, 1976; Schyns & Oliva, 1994). This memory-based knowledge could also be more effective for prototypical scenes including hazards with high (vs. medium) braking affordance. Previous research has shown how scene gist is acquired from information across the entire scene and not only at central vision (Boucart, Moroni, Szaffarczyk, & Tran, 2013; Boucart, Moroni, Thibaut, Szaffarczyk, & Greene, 2013; Boyce & Pollatsek, 1992; Larson, Freeman, Ringer, & Loschky, 2014; Larson & Loschky, 2009; Li, VanRullen, Koch, & Perona, 2002; Thorpe et al., 2001). However, whereas the present data can be explained without referring to gist processing (see above), the assessment of the role of gist in peripheral hazard perception clearly calls for dedicated empirical tests (involving gist previews along with moving window techniques) in future research, probably involving more controlled material such as panoramic scenes with or without hazards at a wide variety of eccentricities. 

Implications for the distinction between visual orientation and hazard processing

The present results further speak to the possibility of a clear separation of visual orientation and hazard processing, based on a decomposition of RTs into (a) the time until the first fixation and (b) the remaining time until the response (Huestegge et al., 2010). The present results clearly suggest that hazard processing (specifically, the processing of hazard level) already starts prior to the initial hazard fixation with the aid of peripheral vision. Thus, the view that hazard processing does not start prior to the first fixation on the hazard is clearly oversimplified. Nevertheless, the distinction between an initial visual orientation process followed by more in-depth processing might still represent a useful heuristic in future research (Nuthmann, 2013, 2014; Nuthmann & Malcolm, 2016). 

Lack of expertise effects on peripheral vision

The lack of any expertise effects on peripheral hazard processing in the present reanalysis corresponds to similar results in the earlier analysis of the data (Huestegge et al., 2010), in which we reported similar durations from scene onset until the first fixation of the hazard region (visual orientation time) for inexperienced and experienced drivers. This is in contrast to previous suggestions that peripheral processing abilities might be mediated by expertise (see, e.g., Crundall et al., 1999; Miura, 1990; Summala, Nieminen, & Punto, 1996; Unema & Rötting, 1990). Instead, it is more in line with another study reporting that driving experience did not influence performance in detecting a closing headway (or red braking lights) in peripheral vision during the fixation of in-car displays (Summala et al., 1998). However, note that Summala et al. (1998) showed that both experts and novices were equally unable to detect hazards with peripheral vision when attending to the in-car display, whereas in our present study both experts and novices were equally able to detect hazards in the periphery. Probably, the lack of expertise effects might be explained by assuming that sufficient traffic scene knowledge has already been acquired even by inexperienced drivers due to their frequent exposure to traffic situations during childhood and adolescence (i.e., prior to their acquisition of a driver's license). 

Whereas previous research reported beneficial effects of driving expertise on general visual processing abilities in traffic (Underwood, Chapman, Brocklehurst, Underwood, & Crundall, 2003; Crundall & Underwood, 1998; Borowsky & Oron-Gilad, 2013; Underwood, Chapman, Bowden, & Crundall, 2002; Underwood, Crundall, & Chapman, 2002) and on hazard perception skills in particular (Grayson & Sexton, 2002; Hull & Christie, 1993; but see Sagberg & Bjornskau, 2006), our results suggest that these benefits are probably not due to marked differences in peripheral vision abilities or due to different levels of knowledge regarding “where to look” for hazards. Instead, the evidence for increased hazard processing duration (i.e., hazard processing during central vision) for inexperienced drivers in our data (Huestegge et al., 2010) are more in line with previous research showing that inexperienced drivers exhibit longer fixation durations on potentially dangerous objects as compared to experts (Chapman & Underwood, 1998; Falkmer & Gregersen, 2005), likely reflecting a deficit in the efficient selection of the adequate (e.g., braking) response. 

General mechanisms underlying efficient peripheral hazard processing

Why is hazard level processed as effectively in peripheral vision as is suggested by our present results? One answer could be that hazards represent potential threats, which might also evoke corresponding emotional responses. For example, a study by Calvo, Nummenmaa, and Hyönä (2007) reported that emotional scenes (which were not related to traffic situations) were preferentially processed over (simultaneously presented) neutral scenes in peripheral vision (> 5° away from central fixation), even though the two scenes were only briefly presented (450 ms). Similarly, Gutiérrez and and Calvo (2011) reported that parafoveally presented prime threat words facilitated responses to probe threat words in comparison with neutral and positive words (at least for high anxiety individuals), suggesting a covert attention bias towards threat stimuli. However, it is also known that other types of information (apart from threat) that are important for humans are processed preferentially in extrafoveal vision, such as direct (vs. averted) gaze of face stimuli (Von Grünau & Anston, 1995) despite comparable overall saliency. Taken together, a general attentional bias towards stimuli of substantial personal relevance might represent a universal mechanism which also underlies the effectiveness of peripheral hazard processing in traffic situations. 

Theoretical implications

Computational models of scene perception (e.g., Itti & Koch, 2000, 2001; Parkhurst, Law, & Niebur, 2002) often assume that activation patterns on a saliency map (Findlay & Walker, 1999) guide fixation locations in scenes. The recently increased interest in naturalistic scenes (e.g., Wolfe, Võ, Evans, & Greene, 2011) led to new search models that are able to explain the scanning of realistic scenes (e.g., Ehinger, Hidalgo-Sotelo, Torralba, & Oliva, 2009; Hwang, Higgins, & Pomplun, 2009; Kanan, Tong, Zhang, & Cottrell, 2009; Navalpakkam & Itti, 2005; Nuthmann, 2014; Pomplun, 2006; Rao, Zelinsky, Hayhoe, & Ballard, 2002; Torralba, Oliva, Castelhano, & Henderson, 2006; Zelinsky, 2008). One crucial issue for these models is the question of the extent of effective peripheral vision. Previous work on determinants of the extent of peripheral vision typically assess the effective field of view by combining moving window procedures (McConkie & Rayner, 1975), which are similar to viewing a scene through a spotlight with increasingly degraded information outside the spotlight (e.g., Caldara, Zhou, & Miellet, 2010; Geisler, Perry, & Najemnik, 2006; Loschky & McConkie, 2002; Loschky, McConkie, Yang, & Miller, 2005; Nuthmann, Smith, Engbert, & Henderson, 2010; Parkhurst, Culurciello, & Niebur, 2000). Findings from these studies suggest that when the display resolution at an eccentricity of about 3° is half of that in foveal vision, the degradation of peripheral vision is no longer detectable (Loschky et al., 2005). However, search performance can still be affected by stimuli located beyond this border in peripheral vision (e.g., Thorpe et al., 2001). Our present results further confirm this view, suggesting that under certain conditions peripheral vision can be remarkably effective, especially when potentially life-threatening information needs to be processed. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Lynn Huestegge. 

Email: [e-mail]. 

Address: Department of Psychology, Würzburg University, Würzburg, Germany.