Luminance cues constrain chromatic blur discrimination in natural scene stimuli

Abstract

Abstract

Abstract:

Abstract  Introducing blur into the color components of a natural scene has very little effect on its percept, whereas blur introduced into the luminance component is very noticeable. Here we quantify the dominance of luminance information in blur detection and examine a number of potential causes. We show that the interaction between chromatic and luminance information is not explained by reduced acuity or spatial resolution limitations for chromatic cues, the effective contrast of the luminance cue, or chromatic and achromatic statistical regularities in the images. Regardless of the quality of chromatic information, the visual system gives primacy to luminance signals when determining edge location. In natural viewing, luminance information appears to be specialized for detecting object boundaries while chromatic information may be used to determine surface properties.

Introduction

A lot is known about the visual processing of chromatic and luminance information. However, rather less is known about how these signals are combined. It is unclear, for example, whether they contribute equally in the localization of visual edges and what happens when they conflict. It has long been believed that color and luminance signals are segregated in the visual system and that edge detection is a predominantly achromatic process (Livingstone & Hubel, 1988). However, recent research suggests that there are neurons in both V1 and V2 that are jointly selective for color and orientation (Gegenfurtner, Kiper, & Fenstemaker, 1996; E. N. Johnson, Hawken, & Shapley, 2001) making it more likely that chromatic information has a role in edge detection. 

It has been argued that pure isoluminant edges are rare in natural images, which would mean that in the majority of cases color is not necessary for the detection of object edges (Zhou & Mel, 2008). However, whilst the majority of edges in natural scenes are a combination of color and luminance, isoluminant edges are not in fact any rarer than achromatic edges, and the contrasts of the components of the combined edges have sufficient variation to be considered independent (Hansen & Gegenfurtner, 2009). This means that isoluminant edges are not inherently any less useful than luminance defined edges. 

Illusions offer striking examples of luminance information appearing to constrain the perceived location of chromatic edges. For example, in the Boynton illusion straight chromatic edges appear to align with nearby irregular luminance edges; the edge location is predominantly determined by luminance information (Kaiser, 1996). Similarly in the Spanish castle illusion chromatic afterimages appear to be constrained by luminance information: Adaptation to a negatively colored image followed by viewing of a blank field produces a blurry, indistinct chromatic afterimage. However, if adaptation is followed by a grey-scale version of the original image it appears to be sharp and normally colored (Sadowski, 2006). Chromatic filling in also appears to be constrained by luminance information, as demonstrated by the watercolor effect (WCE), where color appears to spread between luminance boundaries but does not cross them (Pinna, Brelstaff, & Spillmann, 2001). 

The WCE could suggest that color has a greater role in the perceptions of surface properties as opposed to the perception of edges. For example, color constancy, specifically illuminant discounting and estimation, can be used to facilitate surface segmentation when the stimulus has irregular illumination (see Foster, 2011, for a review of color constancy). Color has also been shown to reduce luminance noise in complex displays, such that dark achromatic targets are unmasked by chromatic variation in the background (Kingdom & Kasrai, 2006). This could be interpreted to mean that chromatic information can be used not only to segment chromatic variation but also facilitate segmentation of luminance information. 

Luminance information also appears to dominate in natural scenes; blurring only chromatic information has little effect on the percept of the scene, whereas blurring luminance information has a profound effect on appearance (Wandell, 1995, figure 7). This phenomenon is not well understood but has been exploited in industrial settings (Isono, Sakata, & Kusaka, 1978; Oho & Watanabe, 2001). In particular analog television systems (PAL, NTSC, SECAM) separate the luminance and two color components and transmit each color at approximately one quarter the resolution of the luminance in order to save bandwidth (Hunt, 2004). 

It is important not only to consider color and luminance information separately but also how they interact with each other. In contrast sensitivity measurements, combining color and luminance information leads to improvements above that which would be predicted by the channels alone (Gur & Akri, 1992). Further, edges in natural scenes may be processed differently depending on whether they are achromatic, isoluminant, or a combination of color and luminance (Kingdom, 2003; Kingdom, Beauce, & Hunter, 2004). 

The above examples have all been used to suggest that luminance information dominates in the visual processing of edges. However, it is not clear whether this is due to a mechanism that gives precedence to luminance information or whether it is due to other factors. For example, chromatic blur may not be visible simply due to poorer spatial resolution in the processing of chromatic information (Mullen, 1985). In the three illusions discussed, and typically in natural scenes, luminance has a higher contrast (Rivest & Cavanagh, 1996), which may also limit detection of chromatic blur. Finally, chromatic and luminance information in natural scenes may have different statistical regularities which could affect how blur is perceived. For example, if the luminance channel contained more high spatial frequency information it could be more susceptible to the blurring process. 

The current study aims to investigate the interaction between sharp luminance information and blurred chromatic information in natural scenes. We quantify the extent to which sharp luminance information masks blurred chromatic information and determine whether this effect results from differential availability of information in the chromatic and luminance channels or from a neural mechanism with a bias toward luminance information. 

Methods—Experiment 1: Blur discrimination

The fact that blur is more obvious when applied to the luminance channel might simply be due to the fact that blur discrimination is poorer for chromatic information. To test if this was the case we examined blur detection for the chromatic information alone and in combination with sharp luminance information. 

Participants

Five volunteers (including the author RJS), with normal or corrected-to-normal vision gave their informed consent and participated in the study. All procedures were approved by the School of Psychology Ethics Committee, University of Nottingham, UK and were in accordance with the Helsinki Declaration. 

Apparatus

A computer-controlled, gamma-corrected, cathode-ray-tube (CRT) monitor was used to present the stimuli. The monitor used was a 19-in Vision Master Pro 454 (Iiyama) with a resolution of 1024 × 768, running at a refresh rate of 85 Hz. A 14-bit digital-to-analog converter (Bits++, Cambridge Research Systems, Cambridge, UK) was used to control stimulus contrast. 

The system was calibrated using a spectroradiometer (PR655, Photo Research, Chatsworth, CA, USA). A chin rest was used to ensure that the participant viewed the stimuli from a constant distance (52 cm). All stimuli were presented and all data were collected using PsychoPy (Peirce, 2007). 

Stimulus generation

The natural images were selected from McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004). The images were from the categories; flowers, animals, foliage, and fruits. We selected the first image in each of these categories that was entirely in focus (with no obvious depth cues), well lit (not predominantly comprised of silhouettes or large areas of darkness), and did not contain text. The central 512 × 512 pixels was then cropped from each image, leading to four equally sized natural images (Figures 2A & 2B). 

Figure 1

 

Creation of stimuli. The chromatic and luminance channels are first separated, then blurred to create the alone conditions (A and B). They are then recombined with their sharp counterpart to create the combined conditions (C and D). Blur detection thresholds were measured for these four conditions. Original image from McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004).

Figure 2

 

Sharp stimuli. The sharp images typically used as foils in each experiment, showing the color manipulations that were made. (A) Original images before any manipulation. (B) Images used in Experiment 1, cropped to 512 ×512 pixels and presented at 50% contrast to allow conversions in color space without exceeding the gamut of the monitor. (C) Examples of images used in Experiment 2, cropped to 512 × 512 pixels, luminance information and chromatic information is presented at five times the participant's corresponding contrast detection thresholds (presented images normalized for participant AP). (D) Images used in Experiment 3, cropped to 512 × 512, luminance and chromatic information has been swapped in MB-DKL space, i.e., luminance information has been replaced with color information and vice versa and presented at 50% contrast. Original images from McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004).

Stimuli were then converted into MB-DKL (MacLeod, Boynton–Derrington, Krauskopf, and Lennie) color space (Macleod & Boynton, 1979; Derrington, Krauskopf, & Lennie, 1984). This transformation from RGB channels to MB-DKL space was performed using the Smith and Pokorny (1975) cone fundamentals and the power spectrum for each gun of the monitor, measured by the PR655 spectroradiometer. In MB-DKL space each image is represented by each pixel's deviation from mean-grey on the luminance axis, the L-M isoluminant axis, and the S-cone isoluminant axis. Each of these channels can then be scaled and/or blurred independently. 

There are two potential issues that could introduce luminance artifacts into the chromatic information. First, we did not adjust the color space transformations according to individual subjects' isoluminance planes. Second, cone adaptation levels can potentially vary across the extent of a natural scene, meaning that using fixed-cone sensitivities (which are implicitly assumed in the MB-DKL space) could introduce luminance artifacts into the color channels (A. P. Johnson, Kingdom, & Baker, 2005). However, if any luminance artifacts were present they would only serve to reduce the effect we were examining. To confirm this we conducted an additional control experiment on one subject using photometrically determined isoluminance, psychophysically determined isoluminance, and a condition with a deliberate luminance artifact (for details please see the Supplementary material). 

All channels were scaled by 50% in order to ensure that the images remained within the gamut of the monitor after chromatic manipulations, with the effect that the overall contrast of the images was reduced. Blurring was performed by filtering the relevant channel(s) with a circular Gaussian filter whose width could be varied with a staircase procedure according to the experimental condition. Either the luminance channel alone was blurred or both of the isoluminant channels (by the same degree). To present the luminance channel alone, the contrast of both chromatic channels were set to zero and, equivalently, to present only chromatic information the luminance contrast was set to zero. After the manipulations had been made the stimuli were converted back to RGB space, for presentation on the monitor. 

Stimuli were presented with a size of 10° of visual angle along each edge, with a gray background. All data collection occurred within a darkened room. 

Procedure

A two-interval forced-choice (2IFC) design was employed. Participants were presented with the two images (foil and target) for 300 ms separated by a 500-ms interstimulus interval (ISI) and asked which appeared more blurred. The presentation order of the foil and target was randomized. 

In each condition we measured the minimal degree of blur that could be detected in this 2IFC procedure, the blur threshold. To determine whether any form of sharp information masks another we measured this blur threshold for chromatic information combined with sharp luminance information, as well as for luminance information combined with sharp chromatic information. Furthermore, to determine whether any differences between color and luminance thresholds are caused simply by poorer blur discrimination, we measured the thresholds for each form of information alone. See Figure 1 for examples of the stimuli in the four conditions. 

The blur threshold in each condition was determined using a one-up, three-down staircase procedure. The staircases controlled the amount of blur in the target images; a different staircase was implemented for each image and these four staircases were randomly interleaved. The staircases were designed to converge on the 79% blur discrimination threshold for each stimulus and aborted after 50 trials. Each participant collected a minimum of two staircases for each image (giving at least eight staircases per condition). Participant RJS collected five staircases for each image, leading to a total of 52 staircases per condition (208 staircases in total). 

Data analysis

Participants' responses were averaged for each blur intensity level presented in the staircase procedure. A Weibull function was then fit to this data. The threshold was derived from this fit as the point at which the observer was at 80% probability of responding correctly. 

Results—Experiment 1

The group data are shown in Figure 3A. A two-way analysis of variance (ANOVA) showed that observers had higher blur discrimination thresholds for chromatic than for luminance information, main effect of channel type, F(1, 76) = 95.664, p < 0.001, MS_channel = 1300.679. Critically, the elevated thresholds for chromatic blur were a great deal more pronounced in the presence of luminance information, interaction between channel and combination, F(1, 76) = 14.548, p < 0.001, MS_interaction = 197.804. 

Figure 3

 

Results. The mean blur detection threshold for both combined (dark grey columns) and single channel (light grey columns) across the group, error bars represent ±1 SEM. (A) Experiment 1, which used the original images (five participants), (B) Experiment 2, contrast-equated images (five participants), and (C) Experiment 3, channel reversed images (10 participants). Hatched columns denote that some staircases were excluded from the analysis as they did not converge, see method for details.

Whilst the blur detection thresholds for the isoluminant stimuli are higher than for the luminance conditions, the thresholds when blurred chromatic information was combined with sharp luminance information were significantly higher again. For the luminance, on the other hand, the presence of sharp chromatic information had no masking effect on the detection of blur. 

Lower acuity, potentially caused by the relative sparsity of S-cones (Wald, 1967), or the low-pass nature of color vision (Mullen, 1985; Parraga, Brelstaff, Troscianko, & Moorehead, 1998), may explain the generally higher thresholds for chromatic blur detection. If these factors were the source of the specific masking effect we found, there would be no difference in the blur detection thresholds between the chromatic blur alone condition and the chromatic blur combined with sharp luminance condition. However, when sharp luminance information is introduced blur detection thresholds are increased again. The increase in blur detection thresholds caused by the introduction of sharp luminance information cannot be explained by poorer acuity of color vision. 

Methods—Experiment 2: Contrast

In natural scenes luminance information has higher effective contrast than chromatic information (Rivest & Cavanagh, 1996) and this may cause it to be a more effective mask. To test whether this explains the effect found above we equated the contrast of the channels according to the individual observers' detection threshold for each (see Stimulus generation below for details). 

Participants

The same participants were used as for Experiment 1. 

Apparatus

The same apparatus was used as for Experiment 1. 

Stimulus generation

The stimuli were initially generated in the same manner as for Experiment 1. In addition, detection thresholds were measured for the luminance and the combined isoluminant channels for each participant for each image using a 2IFC task. The contrast was varied using one-up, three-down staircase procedures and the contrast detection threshold was extracted by fitting a Weibull function to the data from these staircases. Rather than scaling the contrast of each channel by a uniform amount (50%) as in Experiment 1, the channels were each scaled independently for every image and every observer to a contrast that was five times the corresponding detection threshold for that stimulus component (Figure 2C). 

Procedure

The same procedure was used as for Experiment 1. 

Data analysis

The same data analysis was used as for Experiment 1. However, as a result of the lower overall contrast in Experiment 2, 13 (6.25%) staircases had to be excluded as they did not converge; three of these were from the isoluminant condition and 10 were from the blurred chromatic information combined with sharp luminance information condition. 

Results—Experiment 2

The main effect of channel, F(1, 67) = 103.112, p < 0.001, MS_channel = 1503.064, and interaction between channel and combination, F(1, 67) = 14.985, p < 0.001, MS_interaction = 218.418, were entirely undiminished (Figure 3B); the luminance advantage is not caused by the higher effective contrast of luminance information in natural scenes. 

Methods—Experiment 3: Statistical regularities in chromatic and luminance information

Differences in statistical regularities between the channels may also have been the source of the effect. For example the appearance of high spatial frequency stimuli is more affected by blur than low spatial frequency stimuli. Therefore if one channel contains more high spatial frequency information it may be easier to detect blur in that channel. To test if this was influencing the effect we swapped the color and luminance channels, such that chromatic changes in the image became luminance changes and vice versa. If the effect were caused by any difference in the statistics of the information in these natural scenes the effect should also be reversed, causing luminance blur to be masked by sharp chromatic information. 

Participants

Ten volunteers who had not participated in the previous studies, except for the author RJS, took part in this study. All had normal or corrected to normal vision and gave their informed consent. 

Apparatus

A 22-in Vision Master Pro 513 (Iiyama) was used, running at 1280 × 1024, with an 85-Hz refresh rate. The system was calibrated using a spectroradiometer (PR655, Photo Research, Chatsworth, CA, USA) as in previous experiments. A chin rest was used to ensure that the participant viewed the stimuli from a constant 52-cm distance. All stimuli were presented and all data were collected using PsychoPy (Peirce, 2007). 

Stimulus generation

Stimuli were generated in the same manner as for Experiment 1. However, after conversion into MB-DKL space (Macleod & Boynton, 1979; Derrington et al., 1984), the information in the LM and S channels was replaced with the luminance information and information in the luminance channel was replaced with half of the sum of the LM and S information (Figure 2D). 

Procedure

The same procedure was used as for Experiment 1. Each participant collected two staircases for each condition, leading to a total of 80 staircases per condition (320 staircases in total). 

Data analysis

The method of averaging data and fitting a Weibull function could not be performed for all data in this set due to the poor performance levels in the condition combining blurred color and sharp luminance information. For this reason, the simpler method of averaging the final six reversals from the staircase was used. Even then, 31 (9.69%) staircases had to be excluded from the analysis because the subjects' performance was so poor in this condition that the staircase did not converge. Of these, four came from the sharp chromatic information combined with blurred luminance condition and 27 came from the sharp luminance information combined with blurred chromatic information condition. 

Results—Experiment 3

The difference in blur thresholds between the chromatic- and luminance-only conditions was substantially reduced (Figure 3C), to the point that it was no longer statistically significant (Fisher's least significant difference, p = 0.448). However, chromatic blur thresholds remained poor in the presence of sharp luminance information, interaction between channel and combination F(1, 285) = 83.743, p < 0.0001, MS_interaction = 1932.375). Clearly the effect is not caused by differences in the information contained within the chromatic and luminance channels. 

General discussion

A number of previous demonstrations have suggested that luminance information is of particular importance in the detection of visual edges. Here we quantified that dominance using a blur-detection task with naturalistic stimuli and tested a number of candidate explanations for it, namely whether the effect could be explained by poorer chromatic acuity, lower effective contrast, or differences in scene statistics. We found that none of these factors were able to explain the fact that subjects were unable to detect chromatic blur in the presence of sharp luminance information. 

First we showed that differences in acuity are not sufficient to explain the data. Subjects were generally worse at detecting blur in the isoluminant stimuli, which might be ascribed to poorer chromatic acuity, but they were very much worse at the task only when sharp luminance information was combined with the chromatic blur. Even in Experiment 3, for which the modifications to the images resulted in equal blur-detection thresholds for isoluminant stimuli and achromatic stimuli, when the information was combined the chromatic blur became imperceptible. Second, we demonstrated that the effect is not due to the higher effective contrast of luminance information in natural scenes; equating the effective contrast of the channels did not diminish the effect. Third, the effect is not caused by differences in the statistical structure of the color and luminance information; reversing the channels, and therefore the statistical properties of the luminance and chromatic information, did not cause the effect to be reversed or even reduced. 

The fact that chromatic blur alone is harder to detect than luminance blur alone is entirely consistent with previous findings. For instance, studies have shown that blur thresholds for S-cone isolating stimuli are approximately twice as high as those for the other two channels even when cone contrast is taken into consideration (Wuerger, Morgan, Westland, & Owens, 2000; Wuerger, Owens, & Westland, 2001). The reason may be due to reduced spatial sampling of chromatic information leading to a lower precision in chromatic processing (Peirce, Solomon, Forte, & Lennie, 2008). This reduced sampling may, in turn, be a consequence of chromatic aberration; the visual hardware may reflect the lack of spatial precision in the chromatic signals themselves (De Valois & De Valois, 1988). As a result, luminance may be used for tasks requiring high spatial precision. Conversely, color may be used predominately to process surface properties and to facilitate segmentation and grouping, with only a secondary role in edge detection and localization (Mollon, 1989). If color is mainly used to process surface properties this could explain why it appears to be discounted as a cue to edge perception when luminance information is present. 

It is surprising that equating the effective contrast of the color and luminance channels did not reduce the effect. Rivest and Cavanagh (1996) found that luminance does not play a privileged role in a contour localization task if the luminance and chromatic channels are equated to have similar localization thresholds when presented alone. Those authors suggested that the reason luminance appears privileged in natural scenes is due to its greater effective contrast which, at least for the perception of blur, appears not to be the case. 

Color information and luminance information in natural scenes are statistically similar in their 1/f amplitude spectra (Parraga et al., 1998) and in the numbers of achromatic and isoluminant edges that they contain (Hansen & Gegenfurtner, 2009). There might, however, be other statistical differences between the chromatic and luminance information in natural scenes, for example, in the fine structure. Even if natural scenes were not different in general, it might have been the case that the particular images used in this study had different image statistics in the two channels. To ensure that no such statistical artifacts could have caused the effects measured we swapped the information in the luminance and chromatic channels and reran the study. The fact that this removed the advantage for the luminance channel presented alone indicates that there might have been some effect of differential statistics. However, these differences were clearly not responsible for the luminance dominance; when these reversed channels were combined the subjects still gave preference to the luminance channel, even though it now contained no more information than the chromatic channel. Therefore the dominance of sharp luminance information over blurred chromatic information is not related to the statistical structure of natural scenes. At this point the evidence appears to indicate a mechanism giving active preference to luminance signals in the detection of blur. 

It is clear from these data that the signals from chromatic and luminance information are not combined in a simple linear fashion such that it is not sufficient to consider either chromatic or luminance cues in isolation. In the current study we would not have been able to predict the masking effect caused by combining blurred chromatic and sharp luminance information from either the achromatic or isoluminant conditions. The masking effect could only be revealed by testing color and luminance information in combination. Similarly, the phase of a luminance grating overlaid on a chromatic plaid changes the appearance of the plaid (Kingdom, 2003). If the luminance grating is out of phase the plaid has a three-dimensional appearance (an example of the shape-from-shading effect). However, if the luminance grating is in phase with the chromatic information the impression of depth is suppressed. 

The masking effect could indicate that chromatic blur is being bounded by the sharp luminance information, i.e., the chromatic blur does not appear to cross luminance boundaries. When reticles (thin, low-contrast, achromatic lines) are superimposed on the zero crossings of isoluminant gratings this can improve chromatic contrast sensitivity (Montag, 1997). This could be another circumstance where a chromatic gradient is bounded by luminance information. The facilitation effect caused by the reticles may be at the expense of spatial acuity of the chromatic information, i.e., the chromatic information becomes tied to the luminance information. This would mean that the chromatic information would appear aligned with the luminance edges, as seen in the Boynton illusion (Kaiser, 1996, see figure) and the results of the present study. 

There are existing accounts of edge detection such as the scale space (Georgeson, May, Freeman, & Hesse, 2007) and relative phase models (Burr, Morrone, & Spinelli, 1989). However, these do not currently attempt to incorporate the multiple channels (e.g., for chromatic and luminance information) that would be necessary to model the current data. 

Conclusion

The current data show that the process of combining luminance and chromatic signals is not a simple linear summation. When chromatic blur is combined with sharp luminance information chromatic blur detection thresholds are significantly poorer than when presented alone. The converse effect does not occur; blurred luminance information cannot be masked by sharp chromatic information. The luminance masking effect is not caused by poor acuity in the color channels, higher contrast of luminance information, or differences in the statistical properties of the information provided to each channel. This indicates an underlying mechanism that gives precedence to luminance edge information even when more precise chromatic information is available. 

Supplementary Materials

PDF S1 

PDF S2 

Acknowledgments

This work is supported by an EPSRC (United Kingdom) studentship to RJS. 

Commercial relationships: none. 

Corresponding author: Rebecca J. Sharman. 

Email: [e-mail]. 

Address: Nottingham Visual Neuroscience, School of Psychology, University of Nottingham, Nottingham, United Kingdom.