Symmetry impedes symmetry discrimination

Abstract

Objects in the world, natural and artificial alike, are often bilaterally symmetric. The visual system is likely to take advantage of this regularity to encode shapes for efficient object recognition. The nature of encoding a symmetric shape, and of encoding any departure from it, is therefore an important matter in visual perception. We addressed this issue of shape encoding empirically, noting that a particular encoding scheme necessarily leads to a specific profile of sensitivity in perceptual discriminations. We studied symmetry discrimination using human faces and random dots. Each face stimulus was a frontal view of a three-dimensional (3-D) face model. The 3-D face model was a linearly weighted average (a morph) between the model of an original face and that of the corresponding mirror face. Using this morphing technique to vary the degree of asymmetry, we found that, for faces and analogously generated random-dot patterns alike, symmetry discrimination was worst when the stimuli were nearly symmetric, in apparent opposition to almost all studies in the literature. We analyzed the previous work and reconciled the old and new results using a generic model with a simple nonlinearity. By defining asymmetry as the minimal difference between the left and right halves of an object, we found that the visual system was disproportionately more sensitive to larger departures from symmetry than to smaller ones. We further demonstrated that our empirical and modeling results were consistent with Weber–Fechner's and Stevens's laws.

Introduction

Functional vision relies on appropriate representations of object shapes in the world. Among all the shapes in the world, the class of bilaterally symmetric shapes has been considered one of the most important (Baylis & Driver, 1995; Enquist & Arak, 1994; Grammer & Thornhill, 1994; Johnstone, 1994; Kirkpatrick & Rosenthal, 1994; Leyton, 1992; Pennisi, 1995; Thomas, 1993; Thornhill, 1992; Thornhill & Gangestad, 1994; Tyler, 1996; Vetter, Poggio, & Bülthoff, 1994; Wagemans, 1997). How are symmetry and deviation from symmetry (asymmetry) encoded by the visual system? One way to characterize the nature of the underlying perceptual representations is to measure discrimination sensitivity at different degrees of asymmetry. This approach mimics the classic methods to identify mechanisms for encoding luminance contrast (Foley & Legge, 1981; Legge & Folley, 1980). Specifically, for a pair of shapes whose asymmetries were

$p$

and

$p+\mathrm{\Delta }p$

, respectively, observers reported which of the two shapes was more symmetric. We measured discrimination sensitivity

$d\prime$

as a function of pedestal asymmetry

$p$

while keeping

$\mathrm{\Delta }p$

constant. We refer to this as the symmetry discrimination task. 

If bilateral symmetry is not perceptually special, one would expect a flat discrimination function (

$d\prime$

vs.

$p$

) across the entire dimension of asymmetry. If, on the other hand, symmetry is perceptually special, one would expect that symmetry discrimination is either peaked or depressed near perfect symmetry. The profile of the discrimination function therefore reveals how the visual system rescales the external pairwise differences at the various pedestal asymmetries into an internal scale of asymmetry. 

The reasoning above assumes that the pairwise difference

$\mathrm{\Delta }p$

in asymmetry to be discriminated is a constant with respect to some objective scale and independent of pedestal

$p$

. Otherwise, any effect of

$p$

on discrimination sensitivity

$d\prime$

may be confounded by a perceptually covarying

$\mathrm{\Delta }p$

. In fact, this point applies not only to symmetry discrimination, but also to all discriminations in general. However, unlike low-level sensory properties such as contrast, where the unit of physical step

$\mathrm{\Delta }p$

is self-evident and well accepted, there is no universally accepted physical metric of asymmetry. Nevertheless, our results in the paper will show that it is unnecessary to first define or attain a universally accepted metric of asymmetry before the mechanism of symmetry discrimination can be studied and better understood. 

Specifically, we will demonstrate that two seemingly contradictory results, obtained when asymmetry was defined and introduced differently, can be understood by a single and unified account when stimulus characteristics were taken into consideration. To this end, we will demonstrate that a single image-based and biologically plausible model could explain both sets of divergent results with the same set of model parameters. These model parameters, in turn, informed us about the asymmetry metrics used by the visual system. 

The metrics of asymmetry

Mathematically, symmetry is precisely defined—for example, a function

$f(x,y)$

is bilaterally symmetric about the

$y$

-axis when  

$f(x,y)=f(-x,y).$

(1)

 

In contrast, asymmetry is not universally defined, mathematically or perceptually. Perceptually, there were two common ways to introduce and hence measure asymmetry. The first was to embed a perfectly symmetric pattern within a random pattern, for instance, embedding a perfectly symmetric dot pattern within a random-dot field (Barlow & Reeves, 1979). Asymmetry was then quantified as the proportion of symmetric dots among the total number of dots. The second method was to perturb the position of feature points, typically along the direction perpendicular to the axis of symmetry (Freyd & Tversky, 1984), such that a corresponding pair of feature points were no longer perfectly symmetric with each other. This deviation from perfect symmetry, integrated across the entire shape, could be a measure of the degree of asymmetry. 

These two different metrics of asymmetry, however, are not comparable. As will be shown in the next section, these two metrics in fact could yield opposite perceptual results for symmetry discrimination (comparing

$p$

vs.

$p+\mathrm{\Delta }p$

). If one assumes that the visual system executes the same computation when performing a symmetry discrimination task regardless of the metrics of asymmetry used to create the stimuli, one must conclude that at least one of the metrics did not have perceptually uniform steps. That is, perceptually,

$\mathrm{\Delta }p$

varied as a function of

$p$

. This could be a result of how the visual system encoded asymmetry, or could be due to some intrinsic signal and noise properties of the stimuli that had little to do with the visual system, or both. 

From Equation 1, a simple metric of asymmetry of an image

$f(x,y)$

can be defined as  

$\underset{x_0}{\min }\int dxdy\left[f\right(x,y)-f(2x_0-x,y){]}^2,$

(2)

where

$x=x_0$

is the position of the axis of symmetry. Here, asymmetry is explicitly defined as the difference between halves of a retinal image. Because it is image based, this definition can serve as a ruler to compare different metrics of asymmetry with one another. More importantly, this ruler can also be used to measure image-level noise in the stimuli generated using a particular asymmetry metric. Because Equation 2 describes an image property without any assumption about the visual system, the noise thus measured is external to the visual system. This will turn out to be a critical point in interpreting experimental results in this paper. But first, we will look at previous work. 

Previous work

The consensus of most published results appears to be that symmetry was perceptually “salient,” in the sense that human observers were highly sensitive to any deviations from perfect symmetry. A specialized brain mechanism has often been proposed for symmetry perception (Sasaki, Vanduffel, Knutsen, Tyler, & Tootell, 2005; Tyler et al., 2005). This proposal was often taken to support the empirical findings that symmetry discrimination was best near perfect symmetry. Such results were often obtained from studies of symmetry detection, when subtle deviations from perfect symmetry were found easy to detect (Gerbino & Zhang, 1991; Locher & Smets, 1992; Wagemans, van Gool, & d'Ydewalle, 1991, 1992; Wagemans, van Gool, Swinnen, & van Horebeek, 1993). However, when symmetry discrimination was directly studied, the results to date were equivocal. That is to say, when discrimination of a subtle deviation from perfect symmetry has been compared with discrimination of the same deviation from an asymmetry, contradictory results have been reported, depending on how asymmetry was introduced. 

Perhaps the most prominent study was by Barlow and Reeves (1979). They started from a perfectly bilaterally symmetric random-dot pattern and introduced different degrees of asymmetry by replacing a proportion of the symmetric dots by the same number of dots in random positions. By testing at different pedestal asymmetry levels while keeping constant (30%) the difference of asymmetry thus defined, Barlow and Reeves (1979) found that subjects performed best near perfect symmetry. 

In contrast, Freyd and Tversky (1984) found the opposite result using very different stimuli and a different metric of asymmetry (see also King, Meyer, Tangney, & Biederman, 1976). Each stimulus was a totem-like polygon with a central vertical axis and horizontally outgoing limbs. The limbs in each pair were not necessarily equal in length or in height, resulting in fluctuating asymmetry. The metric of asymmetry was defined according to Zimmer (1984), combining both the mismatch in length and in height. 

Recently, Csathó, van der Vloed, and van der Helm (2004) argued that the result by Freyd and Tversky (1984) could be explained by a new metric of asymmetry. This metric, symmetry-to-noise, was defined as the ratio of the number of paired symmetry elements and the total number of elements in a stimulus. Specifically, Csathó et al. (2004) used similar, but not identical, totem-like stimuli as in Freyd and Tversky (1984). They kept constant the height of each pair of corresponding limbs and defined symmetry, in their symmetry-to-noise metric, as the number of paired pixels that was symmetric, and defined noise as the number of mismatched pixels. Csathó et al. (2004) showed that the closer two stimuli were to each other with respect to the symmetry-to-noise metric or the metric of Zimmer's (1984), the more often subjects judged them to be perceptually similar. In fact, this new metric was very similar to that used in Barlow and Reeves (1979). Yet, Csathó et al. (2004) offered no explanation why Barlow and Reeves (1979) found the best discrimination to be near perfect symmetry. In fact, according to the reasoning by Csathó et al. (2004), they would predict equal discrimination in Barlow and Reeves (1979), which was not the case. Later in the current paper, we will computationally demonstrate that, with a stimulus whose asymmetry originated from uneven lengths of pairs of limbs of a totem-like figure, symmetry discrimination could be predicted from the first principles, using Weber–Fechner's law, to be worst for stimuli near perfect symmetry. 

When asymmetry was implemented as unequal distances of pairs of feature points to the axis, as in Csathó et al. (2004), an alternative way to define and manipulate asymmetry is morphing. For simplicity, consider a figure that shapes like a fish bone. It has a central axis with “limbs” (or bones) extending to the left and right. The limbs always come in pairs, but the left and right limbs in a pair need not be equally long. We can represent such a stimulus with two vectors: one we will call

$\mathit{L}$

, being a list of limb lengths arranged from top to bottom for the left side, and the other called

$\mathit{R}$

for the right side. From an asymmetric figure

$<\mathit{L},\mathit{R}>$

, we can construct a family of new figures

$<l,r>$

by morphing (i.e., a weighted sum of the left and right):  

$\begin{array}{c}l=\frac{\mathit{L}+\mathit{R}}{2}+m\frac{\mathit{L}-\mathit{R}}{2},\\ r=\frac{\mathit{L}+\mathit{R}}{2}-m\frac{\mathit{L}-\mathit{R}}{2},\end{array}$

(3)

where

$m\epsilon (-\infty ,\infty )$

is a parameter that controls the shape and the degree of asymmetry of the resulting figure

$<l,r>$

, given the original

$<\mathit{L},\mathit{R}>$

. For instance, when

$m=0$

, we have a perfectly symmetric figure; when

$m=1$

, we have the original figure; and when

$m>1$

, the asymmetry of the original figure is exaggerated. Using one version of this morphing method (Gryphon's Morph software), Rhodes, Proffit, Grady, and Sumich (1998) studied symmetry discrimination of human faces for the pair (

$m=1$

vs.

$m=0.5$

) and the pair (

$m=0.5$

vs.

$m=0$

). They found that subjects were better at discriminating the former pair than the latter, meaning that symmetry discrimination was worse when the stimuli were closer to symmetry. Rhodes et al. (1998) assumed that the physical difference (0.5) was perceptually constant but did not verify this using, for example, Equation 2. They suggested that (

$m=1,m=0.5$

) were better discriminated than (

$m=0.5,m=0$

) because of familiarity of subjects with natural faces (

$m=1$

). 

Overview of experiments

In this paper, we will first replicate, using a different morphing method, the Rhodes et al. (1998) result of symmetry discrimination of faces. We will then demonstrate that this result was not likely due to familiarity, by showing that similar results could be obtained using upside-down faces and random-dot patterns alike. Next, we will use the method of Barlow and Reeves (1979) to generate asymmetry and replicate their result, which apparently contradicted our results (and that of Rhodes et al., 1998) obtained with morphing. We will then show that a single computational model explained these two sets of results both in terms of overall performance and in a trial-by-trial analysis. This successful model in turn informed us of the asymmetry metric used by the visual system. Finally, we will show that the model was in qualitative agreement with Weber–Fechner's and Stevens's laws. 

Experiment 1: Morphed faces

Stimuli

We used 3-D face models to construct synthetic face images with varying degrees of asymmetry within a natural range. These face models were acquired with a Cyberware 3-D Laser Scanner and provided by the Max Planck Institute for Biological Cybernetics in Tübingen, Germany (Blanz & Vetter, 1999). Hair, which could not be accurately captured, was removed from the models. The shape of a face, in terms of spatial coordinates (

$x,y,z$

), and surface color, in terms of RGB values, were captured with an array of 512 × 512 sampling points, with half of the points on either half of the face. The positions of these sampling points were selected by a modified optical flow algorithm (Blanz & Vetter, 1999) operated on the raw digital scans, which in effect matched a feature point on one side of the face to its corresponding counterpart on the other side. 

Two male faces with a highly noticeable degree of natural asymmetry were used to generate the synthetic faces in the experiment. Each face model

$\mathit{O}$

was represented as a dense (512 × 512) array of 3-D coordinates of surface position (

$x,y,z$

) and pigmentation (

$r,g,b$

) (Blanz & Vetter, 1999). For each face model

$\mathit{O}$

, we created its mirror “twin” by swapping the shape and color values between each pair of the corresponding sample points. The degree of asymmetry was manipulated by taking a weighted vector average (i.e., morphing) of the 3-D face

$\mathit{O}$

and its mirror counterpart ; that is, 

$\mathit{F}\left(m\right)=1/2\left[\right(1+m)\mathit{O}+(1-m\left)\widehat{\mathit{O}}\right],$

(4)

except that the surface pigmentation was always that for the perfectly symmetric face. Note that the asymmetry scale defined by

$m$

was specific to each individual. To generate a test image, the synthetic 3-D face model

$\mathit{F}\left(m\right)$

was rendered with perspective projection from a frontal view, with the lighting direction coinciding with the line of sight. Samples of the resulting images from one of the two faces are shown in Figure 1. 

Figure 1

 

Example faces created by differentially weighting a face and its mirror counterpart. The face in the middle was equally weighted, thereby being perfectly symmetric.

Local and incidental features in the image necessarily covaried with the degree of asymmetry. To prevent subjects from making symmetry judgments based solely on salient local features, several masks were applied to each face image. An asymmetric irregular soft mask was added to the hairline and neck-cut contours, respectively, to produce an irregular but smooth transition from the face to the black background. A gray square was placed on the left to occlude the T-junction where the neck met the jaw line. Without this gray square, the relative height of the T-junctions on each side of a face could have been used as a cue for asymmetry judgment. As it will become clear from the qualitatively similar results of Experiments 1 and 3, the irregular hairline and the gray square did not have any idiosyncratic impact on our results. A sample of the final stimuli is shown in Figure 2. The resulting image was 350 × 350 pixels, with the face offset slightly to the upper right. At the viewing distance of 74 cm, a face subtended 6.5° of visual angle (ear-to-ear). 

Figure 2

 

An example stimulus in a trial. Subjects decided whether the face on the left or right side was more symmetric.

Procedure

In each trial, two synthetic faces from the same individual were presented at different degrees of asymmetry. One face's asymmetry was

$m$

(pedestal asymmetry), the other

$m+\mathrm{\Delta }m$

. The two faces were presented either both upright or both inverted. Stimuli stayed on the screen until the subject had responded or until 4 s had elapsed. Subjects viewed the stimuli binocularly and indicated which of the two faces appeared more symmetric by pressing one of two keys. No feedback was provided. Subjects were instructed to be as accurate as possible without concern for response time. 

The values of

$m(-1,0,1),\mathrm{\Delta }m$

(±0.1, ±0.2, ±0.3), and the orientation of the faces (upright vs. inverted) were randomly interleaved. Each combination was repeated 20 times. The total length of an experiment lasted for about an hour, with the total number of trials (720) divided into four blocks so that a subject could take breaks. 

An SGI (Silicon Graphics, Inc.) computer with 24-bit-color graphics system was used to create the stimuli. A 19-in. Sony display with a gamma of 2.0 was placed 74 cm in front of the observer to present the stimuli. 

Subjects

Sixteen paid subjects, aged 18–30 from the University of Tübingen, Germany, participated in the experiment after informed consent. All subjects had aided or unaided Snellen acuity equal to or better than 20/20. They were unfamiliar with the stimuli and were not told how asymmetry in the stimuli was manipulated. Half of the subjects, randomly selected, were presented with the synthetic faces from one individual, the other half of the subjects the other individual. 

Results

Figure 3 shows

$d\prime$

as a function of

$m(-1,0,\mathrm{a}\mathrm{n}\mathrm{d}+1)$

at three levels of

$\mathrm{\Delta }m$

(±0.1, ±0.2, and ±0.3) for upright faces. Figure 4 shows the same pattern of results for inverted faces. 

Figure 3

 

Symmetry discrimination for upright faces, averaged across subjects. Discrimination was worst near perfect symmetry. Left panel shows results with decrements in Δ⁢m; right panel shows results with increments. Error bars represent standard error of the mean.

Figure 4

 

Symmetry discrimination for inverted faces, averaged across subjects. Discrimination was worst near perfect symmetry, similar to the results with upright faces.

With the difference of asymmetry

$\mathrm{\Delta }m$

between the two images kept constant, discrimination was worse for stimuli near perfect symmetry than at the asymmetric original face or its mirror version, regardless of the orientation of the face. [ANOVA, within-subjects design:

$m\times |\mathrm{\Delta }m|\times$

sign(

$\mathrm{\Delta }m)\times \mathrm{o}\mathrm{r}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}$

; significant main effects of

$m$

,

$F(2,26)=25.18$

, p < .0001;

$|\mathrm{\Delta }m|$

,

$F(2,26)=16.1$

, p < .0001; and orientation,

$F(1,13)=10.42$

, p < .005; no significant effect of sign

$(\mathrm{\Delta }m)$

,

$F(1,13)=0.63$

.] 

To ensure that the results above were not due to any artifacts in the test images, we computed the image-level asymmetry for each face stimulus according to Equation 2. We searched for the position of a vertical axis that gave rise to the minimal pixel-level difference between its left and right side in an image. This difference was calculated as the root mean square difference in pixel values (in range of [0,1]), with red, green, and blue channels weighted equally. We excluded from the calculation any pixels that belonged to the occluding gray square and the corresponding pixels on the other side of vertical axis. The minimal difference was greater than zero at

$m=0$

because of the asymmetric soft masks applied to the hairline and the neck. 

It turned out that this minimal image-level difference as a function of

$m$

had a steeper slope near

$m=0$

than away from it (Figure 5). This means that, based on the pixel value differences alone, discrimination should have been best near perfect symmetry, exactly opposite to the human data. In other words, the physical difference in the image, in the absence of any encoding scheme, cannot explain the human data above. 

Figure 5

 

2-D image-level difference, in terms of the minimum Euclidean distance in pixel values, between the left and the right half of a face image as a function of m. The left and right halves were defined by a vertical axis whose location gave rise to the minimal difference.

Experiment 2: Unmatched random dots

Thus far, our results were contrary to what had been found in Barlow and Reeves (1979). In that study, asymmetry was introduced in a different manner. Starting from a perfectly symmetric 2-D pattern of random dots, a certain percentage of the dots was replaced by the same number of dots that were positioned randomly. The degree of asymmetry, which we shall denote by “

$u$

” (as in “unmatched”), was defined as the percentage of the randomly positioned dots. With asymmetry thus defined, it was found that symmetry discrimination was best near perfect symmetry. 

In this experiment, we replicated the result of Barlow and Reeves (1979) using random-dot stimuli. To pave the way for Experiment 3, we generated our stimuli by texture mapping 100 pairs of symmetric dots (200 dots total) onto the surface of a perfectly symmetric 3-D face model (

$m=0$

). We refer to a pattern thus generated as a base pattern. Ten base patterns from 10 symmetric faces were used for this experiment. Images in a trial were always generated from a single base pattern. Starting with a base pattern, asymmetry was introduced by randomly replacing

$u$

proportion of the dots with unmatched dots. Although the random dots were confined to the surface (and thus within the outline) of a face, the resulting stimuli did not resemble faces. Subjects debriefed after the experiment referred to the patterns as “angel,” “airplane,” “dancer,” and “face mask,” etc. 

Three pedestal asymmetry levels (

$u$

= 0.3, 0.6, and 0.9) and three decrements (

$\mathrm{\Delta }u$

= −0.1, −0.2, and −0.3) were tested (whether the delta asymmetry was an increment or decrement was immaterial because a trial with

$u=0.3$

and

$\mathrm{\Delta }u=-0.3$

would be the same as a trial with

$u=0$

and

$\mathrm{\Delta }u=+0.3$

). The orientation of the stimuli was always “upright,” although this factor was irrelevant for these dot patterns. Each pedestal and decrement combination was presented to a subject 80 times, evenly distributed across the 10 base patterns. Figure 6 shows two example stimuli obtained from one base pattern. Six fresh subjects participated in the experiment, which was otherwise identical to Experiment 1. 

Figure 6

 

Two examples of the random-dot stimuli, when asymmetry was introduced by randomly replacing a proportion ( u) of dots in a perfectly symmetric dot pattern. The left pattern has 10% random dots ( u=0.1), whereas the right has 70% ( u=0.7). Both were derived from the same base pattern.

Figure 7 shows the results of this experiment that replicated the original findings from Barlow and Reeves (1979). Namely, symmetry discrimination was better near perfect symmetry. [ANOVA, within-subjects design:

$u\times \mathrm{\Delta }u$

; significant main effects of

$u,F(2,10)=144.8$

, p < .0001; and

$\mathrm{\Delta }u$

,

$F(2,10)=86.69$

, p < .0001.] 

Figure 7

 

Symmetry discrimination of dot patterns, whose asymmetry was defined as in Barlow and Reeves (1979). The curve with circular data symbols represents the original result of Barlow and Reeves (1979) when the difference of asymmetry was 30% (derived from their Figure 4, dotted line). The curves with filled data symbols are from the current study. The discrimination was best near perfect symmetry.

Experiment 3: Morphed random dots

We now return to our morphing method. Here in Experiment 3, we used random-dot stimuli to replicate the pattern of our results from the face experiment in Experiment 1 and demonstrated that our results in Experiment 1 were not face specific. We started with the 10 base patterns used in Experiment 2, each with 100 symmetric random-dot pairs (200 dots total) placed on the surface of a perfectly symmetric face (

$m=0$

). We then introduced asymmetry in the same manner as with the faces in Experiment 1, using the random-dot pattern as a texture mapped onto a 3-D face model, whose shape was altered according to Equation 3. This procedure ensured that the displacements of the random dots, and thus the amount of asymmetry, were adjusted using the same

$m$

-scale for the faces in Experiment 1. Figure 8 shows two example stimuli thus created from the same base pattern. 

Figure 8

 

Two example stimuli, when asymmetry was introduced by morphing in the same way as for the faces in Experiment 1. The left pattern is perfectly symmetric ( m=0), whereas the right pattern is m=1.2.

Six fresh subjects participated. Each subject was tested at all combinations of the pedestal asymmetry (

$m$

) of 0, 0.5, and 1 and increments (

$\mathrm{\Delta }m$

) of +0.2, +0.4, and +0.6. Each pedestal/increment combination was tested 40 times, evenly distributed across the 10 base patterns. The setting of the experiment was otherwise identical to Experiment 2. 

Figure 9 shows the results averaged over the subjects. As with the faces in Experiment 1, discrimination sensitivity improved with increasing amounts of

$m$

-scale asymmetry. Therefore, the results from Experiment 1 generalized to random dots. [ANOVA, within-subjects design:

$m\times \mathrm{\Delta }m$

; significant main effects of

$m$

,

$F(2,10)=30.21$

, p < .0001; and

$\mathrm{\Delta }m$

,

$F(2,10)=23.27$

, p < .0001.] 

Figure 9

 

Symmetry discrimination of dot patterns, when asymmetry was introduced via morphing. Discrimination was worst near perfect symmetry ( m=0), consistent with results of the face experiment (Experiment 1).

A unified computational account

Results of all three experiments above can be summarized as follows. Starting with a symmetric pattern, when asymmetry was introduced with orderly structural deformation via morphing, discrimination was worst near perfect symmetry for faces and random-dots alike. In contrast, when asymmetry was introduced with random replacement of dots, discrimination was best near perfect symmetry. 

Symmetry discrimination, apparently, depended on how asymmetry was generated and quantified. However, the seemingly contradictory results were nevertheless produced by the very same visual system. Can a simple and biologically plausible model explain this pattern of results and thereby inform us on how asymmetry may be processed by the visual system? 

We will first turn to the image properties of the asymmetric stimuli generated by the different methods. We will then propose a simple mechanism and fit a single set of model parameters simultaneously to the opposing results from Experiments 2 and 3. We will consider only the random-dot patterns because applying the

$u$

-scale asymmetry to faces would render them severely grotesque. 

Noise in the stimulus

We wanted to measure the minimal left–right distance in pixel values with the random-dot stimuli as we did with the faces. Due to the sparse nature of the random dots, we first blurred an image of random dots by a low-pass filter (rotationally symmetric Gaussian kernel, SD = 20 pixels, which was 9% of the ear-to-ear distance—the specific size of this blurring kernel did not qualitatively affect our analysis). We then searched for the position of a vertical axis that minimized the left–right Euclidean (RMS) difference in pixel values, as in Figure 5. We plotted this minimal difference as a function of the two scales of asymmetry used in this paper. As shown in Figure 10, introducing asymmetry by morphing yielded an almost linear function with only a weak deceleration:

$D\left(\mathrm{m}\right)=1.93m^{0.82}$

,

$\mathit{R}=0.99998$

; whereas that by random replacement yielded a strongly decelerating curve:

$D\left(\mathrm{u}\right)=5.52u^{0.48}$

,

$\mathit{R}=0.99992$

. 

Figure 10

 

The smallest possible left–right difference in pixel values, D, for the m-scale (left) and the u-scale (right). Each dashed curve represents one family of dot patterns, each of which was generated from one base pattern. Across families, the extent of the non-monotonicity represents the amount of spurious noise inherent in a pattern due to the random placement of dots. The amount of this stimulus noise is characterized by the within-family standard deviations (Loftus & Masson, 1994) along the average values of D (solid curves). Noise increased with asymmetry for the u-scale but not for the m-scale.

For an observer whose decision was based solely on the image-level difference, a linear relationship between

$D$

and asymmetry, as approximated by the

$m$

-scale, would give rise to a constant symmetry discrimination function; whereas a decelerating curve, as with the

$u$

-scale, would give rise to best discrimination near perfect symmetry. Our data with the morphing method, where discrimination was worst for stimuli near perfect symmetry, could not be explained by the image-level properties measured by

$D$

but must be attributed to how the visual system represented the image. On the other hand, the result in Barlow and Reeves (1979) using the random replacement method had as much to do with the physical stimuli as with the visual system. 

We next considered the variability in image-level asymmetry. Each error bar in Figure 10 shows the variations in D (within-family) standard derivation, analogous to the within-subject confidence intervals of Loftus & Masson, 1994) from different random-dot samples at an asymmetry level. This random variation was nearly constant over the entire morphing (

$m$

) scale (

$er{r}_D\left(\mathrm{m}\right)=0.157{\left(m\right)}^{0.12}$

,

$\mathit{R}=0.965$

) and thus affected discrimination almost equally. In contrast, the variation increased in the random replacement (

$u$

) scale as asymmetry increased

$er{r}_D\left(u\right)=1.16u^{0.69}$

,

$\mathit{R}=0.999$

). This was expected because the number of randomly positioned dots increased with asymmetry. This difference in stimulus-level variation, which was a form of external noise, was another reason for the result in Barlow and Reeves (1979) to be dictated by the image properties of the stimuli (higher stimulus noise at higher asymmetry, and thus worse discrimination). 

A computational model

We hypothesized that the human visual system discriminated symmetry in the same way in both cases of asymmetry manipulation, and that the seemingly contradictory results were due only to the interaction between stimulus and the visual system. If this was true, we should be able to derive a single computational model that accounted for both the

$m$

- and

$u$

-scale results. Furthermore, because of the large difference in the amount of external noise between the two methods of introducing asymmetry, we expected that on a trial-by-trial basis, our model's performance should differ for the two types of stimuli and correlate with each individual subject's performance. 

We adopted a type of model that is commonly used in visual psychophysics, which consists of a linear operation followed by a nonlinear transducer function (Figure 11) (Lu & Dosher, 1998; Scognamillo, Rhodes, Morrone, & Burr, 2003; Watson, 1983; Wilson, 1989). Our model computed the minimal left–right difference of a Gaussian-blurred image by finding the optimal placement of the symmetry axis and then raised the computed Euclidean pixel-wise difference to a power of

$\mathit{\alpha }$

. This measure of the image-level difference was then perturbed by an additive internal Gaussian noise

$N(0,\mathit{\sigma })$

, resulting in a noisy measurement of asymmetry. In other words, we assumed that the perceived degree of asymmetry was a noisy nonlinear (if

$\mathit{\alpha }\ne 1$

) measurement of the minimal left–right difference of an image. The only two free parameters in the model were

$\mathit{\alpha }$

 and

$\mathit{\sigma }$

. We presented the model with the stimuli used in Experiments 2 and 3 and adjusted the model parameters to simultaneously minimize the mean square error between the model data and human performance from the two experiments. Figure 12 shows the model's fit to the two sets of seemingly contradictory data, which captured the trend of all the data. This was achieved with a single set of parameters (

$\mathit{\alpha }=2.5,\mathit{\sigma }=2.8$

). Of particular interest (see next section) was the value of

$\mathit{\alpha },\mathit{\alpha }=2.5$

, which means that the model de-emphasized small left–right differences but exaggerated larger one. 

Figure 11

 

The asymmetry-encoding model. The smallest possible left–right difference D of a stimulus was computed first. This difference D was then raised to a power α. Gaussian noise was added to the resultant Dα. The magnitude of the final output was the perceived degree of asymmetry.

Figure 12

 

Model fit to the two data sets obtained from the two different types of asymmetry manipulation, where the power α and the standard deviation σ of the Gaussian noise were the only two free parameters. A single set of model parameters was used to simultaneously fit both sets of data. The best-fitting model was with α=2.5,σ=2.8.

Figure 13 provides insight into how a single set of model parameters (

$\mathit{\alpha }=2.5,\mathit{\sigma }=2.8$

) may account for the two apparently contradictory data sets. Regardless of the asymmetry scale used to generate a stimulus, the degree of asymmetry perceived by the model was based on the image-level asymmetry

$\mathit{D}$

, which was the minimal Euclidean pixel-wise left–right difference of a blurred stimulus. The relationship between

$\mathit{D}$

and the asymmetry scale is replotted in Figure 13 (from Figure 10) for the pedestal stimulus (thick lines) and the pedestal + increment stimulus (thin lines). If there were no nonlinearity in the model (i.e.,

$\mathit{\alpha }=1$

), the signal for symmetry discrimination would be

$\mathrm{\Delta }D$

, which is the vertical separation between the solid lines in each panel of Figure 13. As the pedestal asymmetry increased, signal decreased only slightly for the

$m$

-scale stimuli but much more for the

$u$

-scale stimuli (from

$\mathit{u}=0.3$

to

$\mathit{u}=1.0$

). 

Figure 13

 

Stimulus-level signal and noise for two of the experimental conditions in Figure 12, with Δ⁢m=0.6 (left panel) and Δ⁢u=-0.3 (right panel). The left ordinates of both panels and the solid curves show the minimal left–right difference ( D), which was the common measure of stimulus-level asymmetry used by the model, as functions of m and Δ⁢m (left panel), and u and Δ⁢u (right panel). Error bars indicate the stimulus-level variability (SD) at different pedestal asymmetries (from Figure 10). The right ordinates of the two panels show d' (open circles and dotted curves) of the model as functions of m and u if α=1 and σ=0.

As we have discussed, random placement of the dots in the stimuli perturbed

$D$

(Figure 10). The error bars in Figure 13 depict this stimulus-level noise, which limited performance even if there were no internal noise in the model. For the

$u$

-scale stimuli, stimulus noise increased with asymmetry, and the task became noise limited at high pedestal asymmetry. Stimulus noise was smaller in amplitude and less variable for the

$m$

-scale stimuli. The dotted lines with open circles in Figure 13 depict performance for a linear model without internal noise (i.e.,

$\mathit{\alpha }=1$

and

$\mathit{\sigma }=0$

). In this case,

$d\prime$

decreased with increasing asymmetry for both types of stimuli, but more so for the

$u$

-scale stimuli than for the

$m$

-scale ones. 

Increasing the nonlinearity

$\mathit{\alpha }$

of the model beyond 1.0 exaggerated the asymmetry perceived by the model (

$D^{\mathit{\alpha }}$

) at higher asymmetry. The discrimination signal

$\mathrm{\Delta }\left(D^{\mathit{\alpha }}\right)$

likewise increased with asymmetry. With a sufficiently large

$\mathit{\alpha }$

(2.5 for our data), the discrimination signal could become larger, and the task easier, as asymmetry increased, provided that stimulus noise was not a limiting factor. This is because stimulus noise also increased with

$\mathit{\alpha }$

, and could cancel any gain in the signal. The model's internal noise (

$\mathit{\sigma }$

) determined if performance was limited by stimulus noise—the higher

$\mathit{\sigma }$

was relative to the stimulus noise, the less the effect of stimulus noise would have. At

$\mathit{\alpha }=2.5$

and

$\mathit{\sigma }=2.8$

, the model performed in qualitatively opposite ways (Figure 12) for the two types of stimuli because at this level of internal noise (and nonlinearity), discrimination of the

$u$

-scale stimuli remained limited by stimulus noise whereas discrimination of the

$m$

-scale stimuli was not. As a result, the relatively large

$\alpha$

led to an increasing

$d\prime$

with

$m$

-scale asymmetry, but it had no qualitative effect with the

$u$

-scale stimuli. 

With the model's parameters fixed based on the group data, we also correlated, on a trial-by-trial basis, the model's responses (i.e., the left or right image was more symmetric) with each of the 12 subjects' in the two random-dot experiments (Experiments 2 and 3). This was compared with chance correlation between human and model responses. The chance correlation was obtained by randomly shuffling, within each trial sequence per subject, the trial-by-trial model responses relative to the subject's responses. The trial-by-trial correlation for every subject was without exception statistically greater than chance correlation (Figure 14; Wilcoxon one-tailed test,

$T=0$

, p < .05). Thus, the model not only captured the average results, but also the trial-by-trial variations. 

Figure 14

 

Trial-by-trial correlation of the model with each subject (solid line). Chance correlation, obtained by random shuffling, was also plotted (dash line). The six subjects on the left were in the experiment using the morphing method (“m”, Experiment 2), whereas those on the right the unmatched dots method (“u”, Experiment 3). Trial-by-trial correlation of the model was always better than chance.

According to these modeling results, we concluded that, with respect to left–right image-level differences and excluding any idiosyncratic properties of the different stimuli, human sensitivity to asymmetry was worst near perfectly symmetric stimuli. As is shown in Figures 12 and 14, the proposed encoding scheme of asymmetry (Figure 11) was applicable to the opposing patterns of results in Experiments 2 and 3. 

Weber–Fechner's and Stevens's laws

Our modeling effort strongly suggested that the perceived degree of asymmetry was a noisy nonlinear measure of the minimal left–right difference of an image, which de-emphasized small left–right differences but exaggerated larger ones. 

The fact that small differences were suppressed while larger ones exaggerated might seem to contradict the classic Weber–Fechner's law of sensation, which states that the internal scale of measurement is compressive. We are going to show next that our results were in fact consistent with Weber–Fechner's law, if symmetry was not treated as a basic sensory quantity but was instead derived from other basic sensory quantities. We will also consider Stevens's law, which allows for the internal scale to be either compressive or expansive. We will show that the consistency with our results always held for the Weber–Fechner's law; and within a certain range of the exponent parameter,

$\mathit{\gamma }$

,  for Stevens's law. Our derivation will further show that this

$\mathit{\gamma }$

parameter's range overlaps with the free parameter

$\mathit{\alpha }=2.5$

of our computational model obtained in the last section. The value

$\mathit{\alpha }=2.5$

was obtained by fitting the human data, whereas the following derivation was from a set of first principles without considering any empirical data. 

We note that the following was not an exercise of simple mathematics on a toy problem. In fact, the asymmetry of stimuli used in Csathó et al. (2004) could be completely characterized by the following from the first principle of Weber–Fechner's law. 

We first considered the Weber–Fechner's law. For simplicity and without loss of generality, let us consider only two corresponding dots. Let

$\mathit{L}$

and

$\mathit{R}$

denote the physical distance of the two points from the presumed symmetry axis, respectively. Using our morphing convention, we let

$A=(\mathit{L}+\mathit{R})/2$

and

$B=(\mathit{L}-\mathit{R})/2$

such that this dot pair could generate an infinite number of dot pairs, each pair with a degree of asymmetry

$m$

, as follows:  

$l=A+mB,$

(5)

 

$r=A-mB,$

(6)

where

$l$

and

$r$

are the two new dots' distances to the symmetry axis, respectively. According to Weber–Fechner's law, the perceived length

$y$

of a physical distance

$x$

is  

$y=k\ln \left(x\right)+C,$

(7)

where

$k$

and

$C$

are constant. In symmetry discrimination, we discriminated one pair of dots of asymmetry

$m$

against another pair of dots of asymmetry

$m+\mathrm{\Delta }m$

. Therefore, symmetry discrimination sensitivity

$\mathit{z}$

is determined by  

$z=\frac{d(\mathrm{\Delta }y)}{dm}=2k\frac{S}{S^2-m^2},$

(8)

where

$\mathrm{\Delta }y=k\left[\ln \right(l)-\ln (r\left)\right]=k\ln (l/r)$

and

$S=A/B$

. It is now easy to see the behavior of function

$z(·)$

at

$m=0$

. We compute its first- and second-order derivatives. At

$m=0$

, we have  

$\frac{dz}{dm}=4kS\frac{m}{(S^2-m^2{)}^2}=0$

(9)

and  

$\frac{d^{2}z}{d{m}^2}=4kS\frac{S^2+3m^2}{(S^2-m^2{)}^3}=4k{S}^{-3}>0.$

(10)

 

Thus, we have proved that

$m=0$

is the global minimum of the function

$z\left(m\right)$

. In other words, symmetry discrimination was worst near perfect symmetry. 

The essence of our formulation was not to apply Weber–Fechner's law to the quantity of asymmetry itself. Instead, we applied it to the encoding of stimulus properties about which the degree of asymmetry was to be assessed (i.e.,

$l$

and

$r$

in our example). In this sense, we did not treat symmetry as a basic sensory entity. 

We note that the above derivation could be generalized to the cases where

$\mathit{L}$

and

$\mathit{R}$

are not scalars but high-dimensional vectors. In that case,

$\mathit{z}$

will also be a vector, with each element of it,

${\mathit{z}}_i$

, representing the sensitivity along one dimension of

$\mathit{L}$

and

$\mathit{R}$

. While the visual system may choose to combine these sensitivities in many possible ways, we note that, by applying Equations 8 and 9 to

$z_i$

, these per-dimension sensitivities all reach a minimum at

$m=0$

. While this did not rule out the various special cases, it would be most probable to expect the overall sensitivity to asymmetry also reach minimum at

$m=0$

. 

We now turn to Stevens's law, which states that the sensation

$y$

is a power function of the physical input

$\mathit{x}$

.  

$y=a{x}^{\mathit{\gamma }},$

(11)

where

$a>0$

and

$\mathit{\gamma }>0$

are constant. Similar to the above derivation, denoting

$a{B}^{\mathit{\gamma }}=g$

, when

$m=0$

, we have,  

$\frac{dz}{dm}=0$

(12)

and  

$\frac{d^{2}z}{d{m}^2}=2g\mathit{\gamma }(\mathit{\gamma }-1)(\mathit{\gamma }-2)S^{\mathit{\gamma }-3}.$

(13)

 

Therefore, symmetry discrimination was worst near perfect symmetry when

$\mathit{\gamma }>2$

or when

$0<\mathit{\gamma }<1$

; and best when

$1<\mathit{\gamma }<2$

. Symmetry discrimination would be flat when

$\mathit{\gamma }=2$

or when

$\mathit{\gamma }=1$

. 

We again note that, in our computational model above, the exponent was

$\mathit{\alpha }=2.5>2$

. This indicated that any difference of asymmetry was reduced near perfect symmetry, whereas the same amount of difference was exaggerated away from perfect symmetry. 

Discussion

How is symmetry encoded by the visual system? It might seem that this question is unanswerable without first defining the proper metric of asymmetry. However, this proper metric cannot be known before we know how the visual system encodes symmetry. As if to underline this dilemma, we obtained a pair of seemingly contradictory results using two different methods of introducing asymmetry. When asymmetry was introduced by structural deformation (morphing), symmetry discrimination was worst near perfect symmetry. When asymmetry was introduced instead by random replacement of dots, the opposite result was obtained. 

We first looked into image-level stimulus properties as a result of these two ways of asymmetry manipulation. When asymmetry was introduced by morphing, based solely on the consideration of signal and noise properties of the stimuli, symmetry discrimination should be best for faces and random dots near perfect symmetry (Figure 5; Figure 10, left panel), contrary to the human data. This means that the human results from the morphing experiments could not be explained by properties intrinsic to the stimuli. Instead, these results revealed properties of the visual system. In comparison, we found that the method of introducing asymmetry by random replacement injected increasing amount of stimulus noise as a function of increasing asymmetry (Figure 10, right panel). The human results that symmetry discrimination degraded as asymmetry increased could be attributed to the signal and noise properties of the stimuli. Any observer (biological or computational) will suffer similarly. 

Hence, after taking into consideration the image-level properties of the stimuli, we found that the visual system was in fact less able to discriminate departures from symmetry when the stimuli were close to symmetry. That is, symmetry impeded symmetry discrimination. We uncovered this intrinsic property of the visual system without having to establish the perfect metric of asymmetry. 

We next turned to a simple computational model for a more comprehensive account of symmetry encoding by the visual system. Our simple model estimated the location of a vertical axis that minimized left–right image-level differences. It also suppressed smaller left–right differences while it exaggerated larger ones. We found that this model quantitatively accounted for the two sets of seemingly contrary results with a single set of model parameters (

$\mathit{\alpha }=2.5,\mathit{\sigma }=2.8$

). Because the model suppressed small left–right differences, it suggested that the visual system is inherently insensitive to small fluctuations from perfect symmetry, a conclusion we had also reached with our analysis of the image-level properties of the stimuli. We found that this conclusion was also consistent with Weber–Fechner's and Stevens's laws when they were not directly applied to encode symmetry or asymmetry, but to encode a feature point's distance to the symmetry axis. 

That symmetry was not necessarily treated as a basic sensory entity does not necessarily mean that it was processed inefficiently in the brain. Indeed, given its ecological importance, symmetry may well enjoy highly efficient processing. This was evidenced in a study by Liu and Kersten (2003). These authors found that when the physical difference between stimuli was taken into consideration by an ideal observer (Liu, Knill, & Kersten, 1995), two symmetric objects were better discriminated apart than two asymmetric ones. In other words, discrimination was better between objects that were symmetric than asymmetric. This privileged status of symmetry perception was also indicated by the recent functional magnetic resonance imaging studies by Tyler et al. (2005) and Sasaki et al. (2005). 

In conclusion, the visual perception of bilateral symmetry appears to distort the scale of asymmetry such that any slight departure from perfect symmetry was de-emphasized. Sensitivity to deviations from perfect symmetry was therefore impeded. 

Acknowledgments

We thank Susana Chung, Gordon Legge, Nestor Matthews, Ning Qian, Noah Swartz, and Takeo Watanabe for helpful suggestions. The experiments were conducted at the Max Planck Institute of Biological Cybernetics, Tübingen, Germany, where Tjan was a postdoctoral fellow. Tjan has since moved to University of Southern California. We also thank NEC Research Institute, Princeton, New Jersey, for sponsoring Liu's visit to Tjan in Tübingen that started this collaboration. We thank Heinrich Bülthoff, the editor, and three reviewers for their comments. We also thank Johan Wagemans, one of the reviewers, for pointing out the work by Csathó et al. (2004). 

Tjan's current affiliation is Psychology and Neuroscience Graduate Program, University of Southern California. 

Both authors contributed equally to this work. 

Commercial relationships: none. 

Corresponding author: Zili Liu. 

Email: [e-mail]. 

Address: 1285 Franz Hall, Box 951563, UCLA Department of Psychology, Los Angeles, CA 90095.