Foveal to peripheral extrapolation of brightness within objects

Abstract

Peripheral viewing is characterized by poor resolution and distortions as compared to central viewing; nevertheless, when we move our gaze around, the visual scene does not appear to change. One possible mechanism leading to perceptual uniformity would be that peripheral appearance is extrapolated based on foveal information. Here we investigate foveal-to-peripheral extrapolation in the case of the perceived brightness of an object's surface. While fixating a spot on the rendered object, observers were asked to adjust the brightness of a disc to match a peripherally viewed target area on the surface of the same object. Being forced to fixate a better illuminated point led to brighter matches as compared to fixating points in the shadow, indicating that foveal brightness information was extrapolated. When observers fixated additional points outside of the object on the scene's background, fixated brightness had no effect on the brightness match. Results indicate that our visual system uses the brightness of the foveally viewed surface area to estimate the brightness of areas in the periphery. However, this mechanism is selectively applied within an object's boundary.

Introduction

We perceive the features of a visual scene as stable while we explore it with eye movements despite the fact that peripheral view is characterized by poor resolution (e.g., Hansen, Pracejus, & Gegenfurtner, 2009; Rovamo, Virsu, & Näsänen, 1978; Weale, 1953) and some visual dimensions appear distorted (e.g., Davis, 1990; Greenstein & Hood, 1981; Valsecchi, Toscani, & Gegenfurtner, 2013). It is reasonable to speculate that we interpret unreliable peripheral signals depending on what we sample more reliably in our fovea, yielding to the “illusion of completeness” (Chong & Treisman, 2003). 

There is ample evidence that our visual system puts an emphasis on foveal information. We showed this in the case of lightness judgments, which are biased by the luminance of the fixated areas (Toscani, Valsecchi, & Gegenfurtner, 2013a, 2015). More recently, it was shown that foveal and peripheral information are optimally integrated across saccades (Ganmor, Landy, & Simoncelli, 2015; Wolf & Schütz, 2015). When observers had to judge the orientation of a grating to which they executed a saccade, they based their judgments on both the presaccadic (peripheral) and the postsaccadic (foveal) visual information, weighting the two sources according to their reliability. Further evidence about trans-saccadic integration is provided by studies on object recognition and visual search (Herwig & Schneider, 2014; Herwig, Weiß, & Schneider, 2015). Some lines of research also directly suggest that foveal information informs peripheral vision. For instance Yu and Shim (2016) showed that foveal information influences visual discrimination in the periphery even without awareness. Moreover, postsaccadic foveal feedback calibrates the perception of size between the center and periphery (Bosco, Lappe, & Fattori, 2015; Valsecchi & Gegenfurtner, 2016). 

The case of an object's surface is particularly interesting because it offers two dimensions: lightness and brightness. Brightness varies across the surface, and lightness is constant across the surface. When we look at a shaded surface, uniform in albedo, we perceive luminance variations on its surface while at the same time having a unitary impression about the albedo of the material. This example clarifies the distinction between brightness, defined as perceived luminance, and lightness, defined as perceived albedo. This distinction is supported by the fact that asking people to match the reflectance or to reproduce the luminance of surfaces yields different results (Arend & Spehar, 1993). The same luminance can thus lead to different perceptual reports: one closer to the surface luminance and one closer to the surface reflectance. Our perception of surface albedo appears relatively stable despite the fact that the light reaching the eye of the observer depends also on the surface geometry and the illumination conditions. This phenomenon is called lightness constancy. Constancy is not perfect. Judgments of the lightness of a surface vary with changes in its luminance distribution (Boyaci, Maloney, & Hersh, 2003; Ripamonti et al., 2004; Toscani, Zdravković, & Gegenfurtner, 2016; Zdravković, 2008). Because mistakes in lightness-based object identification are well predicted by brightness (Robilotto & Zaidi, 2004), the computation of brightness has an effect on lightness perception. 

When looking at an object, part of its luminance distribution is sampled foveally—the rest peripherally—therefore, the visual system has to integrate foveal and peripheral information into a unitary lightness percept. Applying a larger weight to foveal sensory input might also serve to limit the effect of the biases that are associated with the peripheral perception of brightness (e.g., Marks, 1966; Osaka, 1980). Here we investigate whether the local brightness of shaded objects at peripherally viewed locations is influenced by the foveal content. In other words, we test the hypothesis that peripheral brightness percepts are partially illusory, being extrapolated from foveal information. 

Some existing observations support this idea. When peripheral brightness perception is tested with simple artificial stimuli, e.g., luminance patches on a luminous or dark background, light patches presented in the periphery of the visual field appear darker than patches presented in the fovea (see Marks, 1966; Osaka, 1980; Zihl, Lissy, & Pöppel, 1980). In natural conditions, however, we do not experience perceptual biases between foveal and peripheral vision. Using a natural scene to test perceptual extrapolation is mandatory also because a relatively rich context is presumed to trigger lightness constancy mechanisms (Gilchrist & Annan, 2002). The tridimensional geometry and the spatial arrangement are essential to provide cues to the visual system to estimate surface albedo (e.g., Adelson & Pentland, 1996; Boyaci, Doerschner, & Maloney, 2006; Boyaci & Maloney, 2004; Gilchrist & Annan, 2002; Pessoa, Mingolla, & Arend, 1996). Notice that in the current study we investigate the effects of luminance at fixation on local brightness matches, but in realistic scenes, lightness and local brightness might be coupled. For example, local brightness matches may be influenced by the interpretation of the scene geometry (Purves, Shimpi, & Lotto, 1999). 

In Experiment 1, we asked our participants to reproduce the appearance of a small area within the surface of an object. They could see the target area only peripherally while fixating a different location on the same object. During the matching procedure, observers could look at the matching disk. However, when they moved their gaze away from the fixation dot on the object, the rendered scene was removed. The results showed that luminance at fixation influenced peripheral brightness matches. In Experiment 2, we demonstrated that the effect of foveal content on peripheral brightness is occurring only within an object's boundary. These results suggest that when it is reasonable to assume a dependency between foveal and peripheral brightness (i.e., within an object's boundaries) our visual system uses the foveal content to interpret peripherally sampled luminance, creating a partially illusory continuity between foveal and peripheral view. 

Experiment 1: Effect of luminance at fixation on peripheral brightness

In this experiment, we tested whether the local brightness (i.e., the perceived luminance on a portion of a surface) in peripheral view is affected by the luminance at fixation. In other words, we tested the hypothesis that when fixating on a surface, the luminance sampled with the fovea influences the perception of the surface luminance seen in the periphery of the visual field. The image of the virtual scene with the target object was shown on the computer only when the participants were looking at the fixation spot, which was removed from the screen when the virtual scene appeared. When they moved their gaze away from the fixation area, the virtual scene was substituted by a checkerboard background and the red fixation dot was shown. Observers were asked to match the luminance of a peripherally viewed selected area on the cylinder surface while we manipulated the luminance at fixation. 

Methods

Observers

Six students from the Justus-Liebig University of Giessen volunteered to take part in the experiment. All observers provided written informed consent in agreement with the Declaration of Helsinki. They all had normal or corrected-to-normal visual acuity, and they were all naïve to the purpose of the experiment. 

Scene description

The scene consisted of a matte cylinder rendered in the center of a gray room with a white matte sphere on its side (Figure 1, left side). The sphere was always the lightest element in the scene in order to prevent any Gelb effect (Gelb, 1929) to affect the target surface. The sphere was rendered with 100% reflectance, and the rendered images were rescaled to have their brightest point as luminous as the brightest luminance produced by the display. Hence, the sphere's brightest point is a reasonable measure of the highest illumination intensity in the scene and corresponded to the white point of the computer screen (113.9 Display Formula\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\bf{\alpha}}\)\(\def\bupbeta{\bf{\beta}}\)\(\def\bupgamma{\bf{\gamma}}\)\(\def\bupdelta{\bf{\delta}}\)\(\def\bupvarepsilon{\bf{\varepsilon}}\)\(\def\bupzeta{\bf{\zeta}}\)\(\def\bupeta{\bf{\eta}}\)\(\def\buptheta{\bf{\theta}}\)\(\def\bupiota{\bf{\iota}}\)\(\def\bupkappa{\bf{\kappa}}\)\(\def\buplambda{\bf{\lambda}}\)\(\def\bupmu{\bf{\mu}}\)\(\def\bupnu{\bf{\nu}}\)\(\def\bupxi{\bf{\xi}}\)\(\def\bupomicron{\bf{\micron}}\)\(\def\buppi{\bf{\pi}}\)\(\def\buprho{\bf{\rho}}\)\(\def\bupsigma{\bf{\sigma}}\)\(\def\buptau{\bf{\tau}}\)\(\def\bupupsilon{\bf{\upsilon}}\)\(\def\bupphi{\bf{\phi}}\)\(\def\bupchi{\bf{\chi}}\)\(\def\buppsy{\bf{\psy}}\)\(\def\bupomega{\bf{\omega}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(cd/{m^2}\)). The physically based renderings were made with the software RADIANCE interfaced with Render Toolbox for MATLAB (Lichtman, Xiao, & Brainard, 2007), which was set to render inter-reflections and penumbrae. The cylinder was rendered with three different albedos (RADIANCE reflectance parameter) in different scenes (35%, 50%, and 65%). The walls of the scene were rendered with a dark albedo (5%) and noise texture. The ambient illumination was simulated by placing a point light source behind the observer's point of view (at 10 times the cylinder height distance from the cylinder center), aligned with the cylinder center on the vertical axis. Another light source was placed on the top left of the cylinder (66° with respect to the scene horizontal center and 4° with respect to the vertical center). The light was placed at about nine times the cylinder height distance from the cylinder center; its intensity was three times higher than the other light source. This second light mainly created a horizontal gradient on the cylinder surface. The position of the light sources caused the objects to cast shadows on the walls of the room; additional luminance contrast was created by the pink noise used in rendering the room walls (RADIANCE function: “dirt dirt.cal -s 1″). Participants sat 38 cm from the center of the computer screen; from this viewing distance, the screen measured 46° × 88° of visual angle and the rendered scene 46° × 48°. The scene was viewed binocularly. The width of the cylinder in the two-dimensional image on the screen was about 20° of visual angle, and its maximum height (the length of its vertical line crossing its center) was about 26°. On the right side of the computer screen, a checkerboard was displayed. The luminance of the white and black squares of the checkerboard was set to the maximum and the minimum of the computer screen, respectively. Observers adjusted the luminance of a gray disk (5° of visual angle radius) that was superimposed on the background on the right side of the monitor. The stimuli and the matching disks in Experiment 1 and Experiment 2 were presented on a Eizo Color Edge CG245W monitor (10 bits per color channel). The gamma curves for each color channel were measured, and the monitor was linearized accordingly. In a pilot study, we confirmed that the current stimuli replicated our previous findings that lightness estimates were biased by fixation location (Toscani et al., 2013a). 

Figure 1

 

Experimental display used in Experiment 1. The virtual scene was presented on the left of the computer screen. A matte cylinder was rendered in the center of the room with a white matte sphere on its left side. On the right side of the screen, a checkerboard was displayed with a superimposed gray disk. Four fixation points were selected, for each side (left and right, LF and DF, respectively), one on the top and one on the bottom of the cylinder surface. The three target areas were selected from the central horizontal line of the cylinder surface (red open circles). Notice that the average luminance in the target areas increased from right to left: 33.9, 17.2, and 5.5 \(cd/{m^2}\); 48.4, 24.5, and 7.8 \(cd/{m^2}\); 63.0, 31.9, and 10.1 \(cd/{m^2}\) for 35%, 50%, and 65% reflectance, respectively.

Fixation points and target areas on the surface

Four fixation points in total were selected: left, right, above, and below the cylinder surface (Figure 1, light fixation [LF] and dark fixation [DF]). The fixation points were selected on the horizontal edges of the cylinder, 3.5° from the edge of each side. Because of the horizontal gradient, the fixation points on the left (LF) were on a brighter part of the surface than the ones on the right (DF). Because the luminance gradient on the cylinder surface was almost exclusively horizontal, vertically aligned points were placed on areas of the surface with the same luminance, but the peripheral context outside the cylinder's surface was different (i.e., floor or back wall). Having fixation positions both above and below the target areas allowed us to control for potential contextual effects due to the surround. 

The target areas were located at midheight in the object, either vertically aligned with one of the fixation points or in the middle of the two (Figure 1, opened red circles: targets). The average luminance in the target areas increased from right to left along the horizontal shading gradient, allowing us to probe the observers' ability to perform the brightness matching task. When the observers were fixating the light fixation point, the targets areas were shown at 8°, 9°, or 14° in the periphery of the visual field, left, center, and right target areas, respectively. When the observers were fixating the dark point, the target areas were 14° (left) and 9° and 8° (right) in the periphery. 

Task

Observers were asked to adjust a uniform circle (5° radius) presented within the right half of the computer screen to match the luminance of a target area on the cylinder surface, indicated by a red open circle (1° radius). The area was indicated during the experiment by a red circle, which appeared when the observers pressed the “a” key on the keyboard. Observers could change the luminance of the disk in the interval between the black and the white of the computer monitor (113.9 cd/m²) by pressing the mouse buttons. One single target area was shown for each trial. The cylinder was rendered with three different reflectances (35%, 50%, and 65%). Observers could see the scene image only when fixating the fixation spot, which in each trial was one of the four previously illustrated (see Figure 1). Observers repeated the matching procedure five times for condition (two horizontal fixation positions, i.e., LF and DF; two vertical fixation positions; three reflectances, and three target areas), giving 180 matches in total. The trials from the different conditions were interleaved. 

Results

Figure 2A represents the average matches as a function of luminance at fixation (fixation: LF and DF) and rendered albedo, showing that when observers were looking at a bright area their luminance matches were, on average, brighter. We performed a four-way, repeated-measures ANOVA to test the effect on the luminance matches of vertical fixation (fixate on the top or on the bottom of the cylinder—same luminance for each couple of vertically aligned points), fixation (luminance at fixation: LF and DF), surface reflectance, and target luminance. 

Figure 2

 

Matching results. (A) Average brightness matches averaged across target areas as a function of the cylinder reflectance. Light gray data points represent the matches in the LF condition whereas dark gray data points represent the matches in the DF condition. (B) Average brightness matches averaged across cylinder reflectance and horizontal fixation condition (DF and LF) as a function of the luminance of the target area. The upward triangles (dark gray) represent the average matches for the conditions in which the fixation point was on the top half of the cylinder; downward triangles (light gray) represent the average matches for the conditions in which the fixation point was on the bottom half. (C) Average brightness matches averaged across cylinder reflectance and vertical fixation position. Light gray points represent the matches for the LF condition whereas dark gray points for the DF condition. (D) Individual average matches used to computed the averages in panel A. The x-axis represents the matches in the DF condition, the y-axis in the LF conditions. Matches for different observers are represented by different symbols. The different colors represent the different surface reflectances from dark to light: 35%, 50%, and 65%, respectively. The dashed black line represents the unity line. Continuous error bars represent the standard error of the distance from the unity line for the three reflectance groups. (A–C) The error bars represent the standard errors of the mean after correcting for between-subject variability (see Franz & Loftus, 2012).

The ANOVA (see Table 1) revealed a main effect of cylinder's reflectance, showing that luminance variation due to the diffuse reflectance rendering parameters was perceivable. The main effects of fixation and target luminance also reached statistical significance but not the main effect of vertical fixation. This result confirms our hypothesis that the perceived luminance of the surface areas that are seen peripherally depends on the luminance in foveal view—at least when the target area and the fixated area belong to the same uniform surface. 

Table 1

 

ANOVA results of Experiment 1. Notes: The table reports the tests on the following main effects and on all their interactions: fixation (horizontal fixation position—luminance at fixation: DF and LF), reflectance (surface reflectance), target (target luminance, increasing horizontally along the cylinder—right, center, and left, respectively), and vertical (vertical fixation position—bottom and top).

 

The interactions between fixation (luminance at fixation) and target luminance and between vertical fixation and target luminance were significant. 

Figure 2B illustrates the interaction effect between vertical fixation (upward triangles and downward triangles: fixate top or bottom, respectively) and target luminance. When observers fixated in the top part of the cylinder, their luminance matches for the brightest target luminance were a little darker than when the observers fixated in the bottom part of the cylinder and vice versa for the matches for the darkest target luminance. When observers fixated in one of the two vertical halves of the cylinder, the context in their visual field changed; e.g., fixating in the upper part determined an overall brighter background. Figure 2C shows the interaction between fixation (luminance at fixation) and target luminance. The effect of luminance at fixation is smaller for the central (middle luminance) target area. If we assume that the effect of foveal luminance on peripherally viewed brightness increases with eccentricity in a linear fashion, this interaction might be the result of the close distance between the central target and the fixation dots as the average distance of the lateral target areas is 11°, and the eccentricity of the central area is 9°. However, the effect of luminance at fixation (fixation) on brightness matches was highly consistent across different observers (Figure 2D). In the LF conditions, matches were brighter than in the DF conditions for all the observers but one, which only showed the fixation effect in the condition in which the cylinder was rendered with the darkest reflectance. 

Experiment 2: Object boundary

In the previous experiment, we tested the hypothesis that peripheral brightness perception is influenced by centrally viewed luminance. The rationale behind this idea is that when there are reasons to suppose a certain degree of uniformity between what is centrally seen and a certain portion of what is seen peripherally, the peripheral percepts might be “filled in” with the foveal information to a certain extent. The potential functional reasons for this filling in are multiple: poor peripheral resolution (i.e., we do not notice the lack of details in peripheral view), distortions in peripheral viewing, and selection of relevant information. It follows that this filling in should occur only when the information in the fovea is somehow predictive of what is present in peripheral viewing. On the other hand, it is also possible that perceived luminance in peripheral viewing is influenced by the luminance seen in the fovea without any assumption of continuity. For instance, an object subliminally presented in the fovea can affect the observer's ability to peripherally discriminate another object (Yu & Shim, 2016). In a second experiment, we tested these two possibilities by forcing people to fixate dark and light areas within and outside of the object's surface. 

Methods

Observers

Nine students from the Justus-Liebig University of Giessen volunteered to take part in the experiment. All observers provided written informed consent in agreement with the Declaration of Helsinki. They all had normal or corrected-to-normal visual acuity, and they were all naïve to the purpose of the experiment. 

Scene composition and target area

We rendered a new scene with a tiled back wall composed of 20 tiles in a 5 × 4 arrangement. The tiles were rendered light or dark gray, arranged in a checkerboard pattern (Figure 3). Observers were forced to fixate one dark (DF, Background) and one light tile (LF, Background) in the background in addition to a dark (DF, Object) and a light (LF, Object) point on the cylinder's surface. The tiles were shaded objects like the cylinder, and the two fixated tiles were matched in luminance with the light and the dark fixation points on the cylinder. Each tile was a rendered three-dimensional object based on the same OBJ (Wavefront Technologies) model. We created the model with MATLAB by defining its three-dimensional geometry with the following equation: Display Formula\(z = a\left| {\sin ({x \over S}\pi )\sin ({y \over S}\pi )} \right|.\). From the perspective of the observer, let x: [0 s], y: [0 s], z: [0 s] be the horizontal, vertical, and depth dimensions of the tile model, respectively, with s being the side of the square and a the depth amplitude:Display Formula\(a = {1 \over 4}s\). The rationale for having observers fixate within the cylinder or on the background was that in the first case it is reasonable to infer some correlation between the foveal and the peripheral luminance (assuming that the cylinder has uniform albedo) whereas this is not true in the latter case. The only target area was centered horizontally on the cylinder and at ∼2° from the top. All of the experimental conditions were also tested with the scene presented upside down. This was done to avoid confounding the manipulation of fixation inside or outside of the object with the hemifield (upper or lower) in which the target area was presented. The ambient illumination was again simulated by placing a point light source far away from the cylinder, behind the observer (at about five times the cylinder height distance from the cylinder center). Another light source was placed in front of the cylinder and on its top left (45° with respect to the scene horizontal center and 45° with respect to the vertical center). The light was placed at 2.3 times the cylinder height distance from the cylinder center; its intensity was 10 times higher than the other light source. This light created an oblique gradient on the cylinder surface so that it was not immediate to extrapolate the luminance of the target area from the fixation points on the cylinder surface. The scene was rendered smaller (30° × 46°) than in Experiment 1 because we decided to present the matching disk peripherally, and therefore, we needed more space on the screen (see the Task and matching disk paragraph). Because of this, the cylinder's sizes were also smaller than in Experiment 1 (12° × 17°). Likewise in Experiment 1, participants sat 38 cm from the center of the computer screen, and the scene was viewed binocularly. 

Figure 3

 

Display used in Experiment 2. Rendered scene with checkerboard on the left, matching disk on the right. The background of the scene consists of a gray wall with a superimposed 5 × 4 tile pattern. The matching disk was visible only when the observer was looking at the central red dot, and the rendered scene was visible only when the observer was looking at the fixation dot within the scene. Hence, the matching disk and the rendered scene were never presented together like they are in this illustration. In the experimental trials, there were two possible fixation points on the cylinder surface and two on the background, one for the DF and one for the LF conditions. The red open circle indicates the target area as it was shown to the observers when they pressed the “a” key. In this illustration, the yellow dots indicate the fixation positions in the catch trials, which were shown in red to the participants. Only one fixation point in the scene was shown in a single trial when the participant was not looking at it, and therefore, the scene was not displayed on the screen. In order to visualize the matching disk, participants had to fixate the red fixation dot on the right of the scene, and it disappeared at fixation.

Fixation points

The two fixation points on the cylinder were placed on its lower part, one 3° away from the left edge and one 3° away from right one. The vertical position of the fixation points was chosen so that the target area was 12° away from each of the fixation points. The size and position of the tiles were chosen so that the fixation points on the background coincided with the center of one light and one dark tile. These two tiles were placed at the same distance between the two fixation points on the cylinder surface and the target area and vertically aligned with them. The tiles were rendered with two arbitrary different reflectances (100% and 35%); then the luminance of all the checkerboard was linearly scaled so that the fixated tiles were matched in luminance with the surround of the two fixation points on the cylinder's surface. It follows that three different scaling factors were computed for each of the three cylinder reflectances. 

Task and matching disk

In addition to the four fixation points that defined the LF and DF conditions within (Object) and outside the object's border (Background), we introduced 18 additional fixation points (Figure 3). These additional fixation points were placed on the centers of the other tiles, on other lateral portions of the cylinder surface, on the cylinder center, and on the target area. These fixation points were introduced in order to make the task nontransparent to the participants and occurred in 36% of the trials, which were not included in the analysis. In order to avoid strategic biases, observers were told that the content of the scene was there to distract them from the task and they had exclusively to focus on the target area and reproduce its luminance in the matching disk. Like in the previous experiments, we used a gaze-contingent display to ensure that observers could see the scene image only when fixating the instructed fixation spot. Contrary to Experiment 1, in Experiment 2 the matching disk was presented peripherally: To visualize the matching disk, observers had to look at a fixation dot presented on the right of the rendered scene, approximately on the center of the computer screen. The position of this fixation point and the matching disk were chosen so as to make sure that the retinal location of the matching disk was adapted to dark while the observers were looking at the scene. This was done in order to avoid any local adaptation effects (see Toscani et al., 2013a, supplementary materials). Observers repeated the matching procedure five times for condition: two fixation positions on the background and two on the object (i.e., LF and DF), three reflectances, and two orientations of the scene (i.e., upright or upside down), giving 120 matches in total, excluding the catch trials. The trials from the different conditions were interleaved. 

Results

Figure 4 represents the brightness matches for the Background and for the Object conditions. When observers where fixating on the object, the matches were positively correlated with the luminance at fixation whereas when the observers were fixating on the background the luminance at fixation did not have any impact on the brightness matches. 

Figure 4

 

Brightness matches for Experiment 2. (A) Matches for the Object condition (y-axis) as a function of the cylinder reflectance (x-axis). Light gray data points represent the matches in the LF condition, black data points in the DF condition. (B) Matches for the Background condition (y-axis) as a function of the cylinder reflectance (x-axis). Light gray data points represent the matches in the LF condition, black data points in the DF condition. Data are averaged across scene orientation (upright and upside down). (C) Individual participants' data: differences between the average matches in the LF condition and the DF condition for each participant. Matches are averaged across target reflectances; black squares represent differences in the Object condition and gray squares in the Background condition. Continuous error bars represent the standard error of the distance from the unity line for the two conditions. (A–B) The error bars represent the standard errors of the mean correcting for between-subject variability (see Franz & Loftus, 2012).

We analyzed the brightness matches with a four-way, repeated-measures ANOVA with cylinder reflectance (30%, 40%, and 50%), fixation (DF and LF), scene orientation (upright and upside down), and fixation position (Object and Background) as factors. The ANOVA (see Table 2) showed a significant main effect of the cylinder reflectance, meaning that observers could see the luminance differences of the target area in peripheral vision. This factor did not interact with any of the other factors. The ANOVA also revealed a significant interaction between fixation position and fixation; i.e., the effect of the fixation disappeared in the Background condition. The effect of scene orientation was not significant, and this factor did not significantly interact with fixation position nor with fixation. The interaction between scene orientation, fixation position, and fixation was also not significant, suggesting that differences within the visual field do not play a role in the results we report here. Figure 4C represents the individual differences between the average matches in the LF condition and the DF condition. The effect of luminance at fixation in the Object condition was variable in size but consistently present in eight out of nine observers. Conversely, in the Background condition, only four out of nine observers produced lighter matches in the LF condition. 

Table 2

 

ANOVA results of Experiment 2. Notes: The table reports the tests on the following main effects and on all their interactions: reflectance (cylinder reflectance), object (fixation position—on the object or on the background), orientation (scene orientation—upright or upside down), and fixation (fixated luminance—DF or LF).

 

Discussion

In the present study, we tested whether the peripheral appearance of a shaded object's brightness is extrapolated based on foveal information. We previously demonstrated that the overall appearance of an object's surface albedo depends on the luminance of the fixated locations (Toscani et al., 2013a, 2015; Toscani, Valsecchi, & Gegenfurtner, 2013b). Here we showed that the luminance at fixation also has an effect on the perceived luminance, i.e., brightness, but only of the portions of the same object's surface viewed peripherally. This finding is compatible with the hypothesis that peripheral view is filled in with foveal information. With an additional experiment, we demonstrated that this filling-in mechanism is selectively applied within an object's boundary. We propose that (a) objects are segmented in peripheral vision and (b) foveally perceived brightness is propagated into the periphery with a filling in mechanism that influences peripheral brightness and terminates at the object boundaries. This process of extrapolation promotes the perceptual uniformity within the boundaries of objects, possibly contributing to visual stability. 

Lightness, brightness, and object boundaries

Lightness is defined as the perceived albedo of a surface whereas brightness is the perceived luminance of a surface (Arend & Spehar, 1993). There is ample evidence that lightness perception highly depends on a surface's luminance distribution (Boyaci et al., 2003; Ripamonti et al., 2004; Robilotto & Zaidi, 2004; Toscani et al., 2016; Zdravković, 2008) and involves complex computations (Boyaci et al., 2006). Specifically, lightness dissimilarity judgments for real objects were explained in terms of brightness (Robilotto & Zaidi, 2004). Here we provide evidence suggesting that the perception of brightness and lightness are intertwined. The results presented here demonstrate an effect of the luminance at fixation on the perceived brightness of peripherally viewed areas of the object. Notice that our results necessarily imply lightness as a determinant of perceived brightness because fixated brightness only influences peripheral brightness as long as the visual system can assume the uniformity of surface albedo, i.e., within the boundary of an object. The fact that brightness and lightness are mutually dependent limits the possible models of brightness and lightness perception. Specifically, strict two-stage models in which lightness is computed after brightness (see Gilchrist, 2015, for a critical review) would not admit any computation of lightness to affect brightness. A model in which both brightness and lightness are independently computed based on an implicit representation of luminance would equally fail to account for our findings. 

Note that our paradigm is probably not the best for showing contextual effects on brightness perception. For instance, stereo information can promote lightness constancy (e.g., Kitazaki, Kobiki, & Maloney, 2008); therefore, our display consisting of binocularly viewed rendered pictures—with no stereo information—is likely to favor local brightness reports as opposed to matches based on the computation of lightness. Additionally, observers were explicitly instructed to ignore the whole scene and focus on the target area. Nevertheless, their brightness matches were consistently influenced by the fixated luminance in an object-dependent fashion. 

In general, our results show that the global appearance of an object's surface is biased by the luminance at fixation. This could explain our previous finding that areas selected by fixations (i.e., the brightest areas of a surface) receive more weight in lightness computation than the portions of an object that are peripherally viewed. 

It is known that top-down factors influence lightness perception in scenes. For instance, Knill and Kersten (1991) demonstrated that different geometrical interpretations of a scene can yield to dramatically different lightness percepts. In their example, the same luminance boundary can be perceived as a lightness edge if the surface is interpreted as flat or as the border of an attached shadow if the surface is interpreted as curved. The influence of the scene interpretation on lightness and brightness perception is well documented in the perceptual literature (e.g., Anderson, 1997; Anderson & Winawer, 2005; Benary, 1924; Gilchrist, 1977; White, 1979; Zdravkovic, Economou, & Gilchrist, 2012; for a comprehensive review, see Adelson, 2000, and A. Gilchrist, 2006). Furthermore, previous research has shown that lightness perception can be modulated by the segmentation of the scene into objects and background, which is closely related to our finding that the effect of fixated brightness is confined within the boundaries of an object. The cues promoting the segregation of foreground and background can even modulate an apparently low-level effect such as simultaneous contrast (Laurinen, Olzak, & Peromaa, 1997). Mechanisms leading to the identification of object boundaries are taken into account in the retinex theory (Land & McCann, 1971). The retinex model computes lightness based on luminance edges associated with albedo changes between surfaces while discounting shallow luminance gradients associated with shading effects. The original retinex theory distinguishes illumination changes and reflectance edges (i.e., boundaries between different objects) in a bottom-up fashion based on edge sharpness. A more modern derivation of the retinex theory, the edge-integration model (Rudd, 2014), incorporates top-down knowledge about object boundaries. Edge integration is proposed to happen at an early visual stage, modulated by top-down feedback that carries information about object boundaries from higher visual areas. 

Our results suggest that the brightness perceived in central vision propagates into the periphery up to the foveated object's boundaries. Notice that previous evidence confirms that peripheral object segmentation can take place despite the poor resolution of peripheral vision. In fact, Thorpe, Gegenfurtner, Fabre-Thorpe, and Bülthoff (2001) demonstrated that natural object recognition is possible even at eccentricities much larger than the ones we tested (i.e., 70.5° of visual angle). 

We do not claim that the interplay between central and peripheral vision could generally explain contextual effects on perception. In fact, there are clear examples of contextual effects that cannot be explained by the fixation behavior. For instance, despite Golz (2010) demonstrated a mediating effect of the viewing behavior on the influence of the surround on the perceived color of a central patch, this surround effect cannot completely be explained by the fixation patter (Granzier, Toscani, & Gegenfurtner, 2012). Additionally, we were able to influence the perceived lightness of an object by shifting covert attention toward lighter or darker areas of its surface while holding fixation constant (Toscani et al., 2013a). 

Peripheral and foveal information integration

At larger eccentricities, perception progressively becomes both less precise and more biased. For instance, color discrimination becomes poorer with eccentricity (e.g., Hansen et al., 2009), in peripheral vision colors appear as desaturated and slightly different in hue (e.g., Gordon & Abramov, 1977; McKeefry, Murray, & Parry, 2007), and contrast sensitivity is reduced (e.g., Rovamo et al., 1978). Additionally, in photopic vision, brightness is underestimated in the periphery (e.g., Greenstein & Hood, 1981). This underestimation bias occurs, for example, when foveal and peripheral simple stimuli are presented in isolation and compared in terms of brightness. In this case, the two stimuli are not perceived to belong to the same surface, and foveal and peripheral brightness are not integrated. We argue that peripheral perception is recalibrated based on the foveal content in order to correct for poor resolution and distortions. It follows that this filling in mechanism should tend to apply to a larger extent at higher eccentricities. Our data are consistent with this hypothesis: In Experiment 2, the effect of luminance at fixation on brightness matches was smaller for the central target area, which was viewed at 9° eccentricity, compared to the two lateral areas, which were, on average, viewed at 11° and, depending on the fixation condition, could be presented at up to 14° eccentricity. This result is consistent with the finding that foveal and peripheral information are integrated across saccades and receive different weights according their reliability (Ganmor et al., 2015; Wolf & Schütz, 2015). 

In Experiment 1, observers tended to overestimate the luminance of the target areas in both DF and LF conditions (see Figure 2C). We believe that this bias emerges because we used an asymmetric matching paradigm, with which the matching disk and the target area are embedded in different contexts, rather than due to a tendency to perceive higher brightness in the periphery. The presence of a bias due to the asymmetric task was confirmed when we computed the average matched luminance from the catch trials (from Experiment 2) in which the fixation point coincided with the target area (i.e., the target area was presented foveally). The matched luminance averaged across cylinder reflectances was 34.8 Display Formula\(cd/{m^2}\), which is higher than the average in the DF conditions and lower than the one in the LF conditions (33.4 and 39.1 Display Formula\(cd/{m^2}\), respectively). Hence, the center–periphery bias in our data is a side effect of asymmetric matching and does not conflict with the underestimation of brightness in peripheral view reported by Greenstein and Hood (1981). 

Perceptual filling in

Our data suggest the visual system uses the perceptual information sampled foveally to fill in the peripheral view when the perceived information is poor and potentially distorted. Mechanisms of perceptual filling in have been extensively described in the past. Lashley (1941) reported that he first noticed his migraine scotoma when he saw his friend's head disappeared and was replaced by the vertical stripes of the background. Several examples of filling in are described in the clinical literature (e.g., Gerrits & Timmerman, 1969; Williams & Gassel, 1962), the lack of vision in the blind spot or scotomas is not noticed, and the missing information from that area of the visual field is replaced by the content of the surround. Similarly, when retinally stabilized stimuli fade, the content of the surrounding part of the visual field fills in their area (Gerrits, De Haan, & Vendrik, 1966). Analogous results are obtained with steady eccentric fixation (Ramachandran & Gregory, 1991). 

In general, it seems that when visual information is missing from a certain part of the visual field, the visual system fills in the perceptual gaps with the content perceived by the portions of the visual field that are most informative about the missing parts. Filling in phenomena are often constrained by luminance and color borders (e.g., Anstis, 2010; Caputo, 1998; Paradiso & Nakayama, 1991; Pinna, 1987; Pinna, Brelstaff, & Spillmann, 2001). Most of the time, the visuals system will rely on the surrounding regions, but when the existence of long-range correlations can be inferred, such as within an object's boundaries, more distant regions can probably play a role. Overall, this mechanism contributes to the perception of a continuous visual scene despite perceptual gaps. 

The effect we discovered here is functionally similar: Where vision becomes poorer, the visual system takes samples from another portion of the visual field where perception is more reliable and complements the degraded information. This process takes place even in the absence of a complete perceptual gap to be filled in. 

Another visual dimension where foveal-to-peripheral extrapolation might play a role is the appearance of texture. Otten, Pinto, Paffen, Seth, and Kanai (2017) found evidence in favor of extrapolation, resulting in the “uniformity illusion.” They had participants fixating on the center of a simple visual display in which the parafoveal content differed from the peripheral one, for instance, texture displays in which the shape or the orientation of the individual elements differed between the center and the periphery or uniformly colored displays with a color shade difference between the two regions. Despite these differences, the displays tended to appear as uniform as if the peripheral content were reconstructed based on the centrally perceived information. Notice however that uniform regular textures, such as the ones that produce the “honeycomb illusion” can also appear nonuniform due to peripheral loss of details (Bertamini, Herzog, & Bruno, 2016; Ninio & Stevens, 2000), indicating that the foveal-to-peripheral extrapolation is not taking place. Our results suggest that, in the domain of brightness, foveal-to-peripheral extrapolation can take place within the context of a naturalistic stimulus. 

Conclusions

Here we demonstrated an effect of luminance at fixation on peripherally perceived brightness. This effect is confined within an object's boundaries. This confirms our previous results that the luminance of a fixated area determines the perceived lightness of an object. Furthermore, it also shows that once the perceived lightness of an object is established, this affects perceived brightness throughout the whole object surface. The asymmetry between fovea and periphery is a fundamental aspect of the architecture of the visual system. It is likely that the extrapolation mechanism for brightness perception is a general principle of vision, and further investigations should test to what extent it applies to other domains of visual perception. 

Acknowledgments

This work was supported by Deutsche Forschungsgemeinschaft SFB TRR 135, by the EU Marie Curie Initial Training Network “PRISM” (FP7—PEOPLE-2012-ITN; grant agreement 316746) and by BMBF-NSF grant 01GQ1304. We specially thank Nicola Bruno for his constructive and expert contribution as a peer reviewer. 

Commercial relationships: none. 

Corresponding author: Matteo Toscani. 

Email: . 

Address: Abteilung Allgemeine Psychologie, Justus-Liebig-Universität Giessen, Giessen, Germany.