A biologically plausible model of human shape symmetry perception

Abstract

Symmetry is usually computationally expensive to encode reliably, and yet it is relatively effortless to perceive. Here, we extend F. J. A. M. Poirier and H. R. Wilson's (2006) model for shape perception to account for H. R. Wilson and F. Wilkinson's (2002) data on shape symmetry. Because the model already accounts for shape perception, only minimal neural circuitry is required to enable it to encode shape symmetry as well. The model is composed of three main parts: (1) recovery of object position using large-scale non-Fourier V4-like concentric units that respond at the center of concentric contour segments across orientations, (2) around that recovered object center, curvature mechanisms combine multiplicatively the responses of oriented filters to encode object-centric local shape information, with a preference for convexities, and (3) object-centric symmetry mechanisms. Model and human performances are comparable for symmetry perception of shapes. Moreover, with some improvement of edge recovery, the model can encode symmetry axes in natural images such as faces.

Introduction

Mirror symmetry, henceforth abbreviated as symmetry, is an important cue guiding mate selection and other behaviors in many species (Horridge, 1996; Møller, 1992; Swaddle & Cuthill, 1994). As Enquist and Arak (1994) argued, animals and insects may have coevolved symmetrical patterns and symmetry perception to improve recognition independent of position and orientation in the visual field. Symmetry is also useful for object processing. Indeed, along with elongation, it helps determine the reference frame for objects (Large, McMullen, & Hamm, 2003; Sekuler & Swimmer, 2000; see also Herbert, Humphrey, & Jolicoeur, 1994). 

Here we further the understanding of symmetry perception by extending an existing model of shape perception (Poirier & Wilson, 2006). This biologically plausible model is efficient, robust, and invariant with respect to shape perception over position and size. Before presenting the model, however, we review relevant aspects of research in symmetry perception (e.g., other models of symmetry, symmetry perception of the outline or shape, and biological mechanisms of symmetry perception). 

Computation of symmetry

Humans recover symmetry in shapes and random textures within 100 ms (Barlow & Reeves, 1979; Carmody, Nodine, & Locher, 1977; Tyler, Hardage, & Miller, 1995; Wilson & Wilkinson, 2002; for reviews, see Tyler, 2002; Wagemans, 1997). Symmetry perception is both sensitive and robust to several perturbations, including percent coherence and spatial jitter of elements composing the symmetrical pattern (Barlow & Reeves, 1979), shifts in midpoint collinearity (Jenkins, 1983), and phase variations (Dakin & Hess, 1997). Symmetry perception is also fairly resistant to slant or skew (i.e., rotation in depth or viewpoint; van der Vloed, Csathó, & van der Helm, 2005; Wagemans, Van Gool, & d'Ydewalle, 1991, 1992; Wagemans, Van Gool, Swinnen, & Van Horebeek, 1993), polarity changes (Mancini, Sally, & Gurnsey, 2005; Tyler & Hardage, 2002; Zhang & Gerbino, 1992), peripheral presentations (Barrett, Whitaker, McGraw, & Herbert, 1999; Julesz, 1971; Mancini et al., 2005; Poirier & Gurnsey, 2005; Sally & Gurnsey, 2001; Tyler, 1999; Wilson & Wilkinson, 2002), temporal perturbations (van der Vloed, Csathó, & van der Helm, 2007), and partial occlusion. 

Many models of symmetry employ some variation where luminance or contrast properties of the image are compared pointwise on either side of a potential axis of symmetry. For texture symmetry, most models are variants of filter–rectify–filter approaches (Dakin & Hess, 1997; Dakin & Watt, 1994; Gurnsey, Herbert, & Kenemy, 1998; Rainville & Kingdom, 1999, 2000, 2002), but other image-filtering approaches have been proposed (e.g., Bonneh, Reisfeld, & Yeshurun, 2002; Palmer, 1983; Royer, 1981) as well as more quantitative approaches (Dry, 2008; van der Helm & Leeuwenberg, 1996, 1999, 2004). Image properties on either side of the potential axis of symmetry are compared using a correlation, a difference, or some related measure (e.g., Dakin & Watt, 1994; Dry, 2008; Gurnsey et al., 1998; Hong & Pavel, 2002; Latimer, Joung, & Stevens, 2002; Mancini et al., 2005; Rainville & Kingdom, 2000). This operation is repeated at all visual locations and all orientations, making this a computationally expensive operation. The problem is compounded by the robustness and flexibility of symmetry perception in natural images, where viewpoint varies, occlusions occur, and light sources vary. 

Computational efficiency and robustness can be increased by focusing the symmetry operations on salient features, such as low spatial frequencies or groups of dots and blobs (Barlow & Reeves, 1979; Julesz & Chang, 1979; Labonté, Shapira, Cohen, & Faubert, 1995) provided the fine-scale structure is not removed in the process (Csathó, van der Vloed, & van der Helm, 2003; van der Helm & Leeuwenberg, 1999, 2004). Indeed, according to ideal observer simulations, about 25% of dots in a random-dot texture are used in the task (Tapiovaara, 1990), possibly those that occur in clusters. Moreover, symmetry perception in low dot-density textures is more efficient and more robust to changes in polarity and distance from foveal center (Tyler & Hardage, 2002; Wenderoth, 1996). Robust shape and symmetry perception can also occur with greater efficiency if processing is based on higher level features, such as object parts and global shape (see Discussion section). 

Center vs. outline

Research indicates that we are more sensitive to symmetry information located in two regions: the first is located near the axis of symmetry (Barlow & Reeves, 1979; Bruce & Morgan, 1975; Jenkins, 1982; Wenderoth, 1995, 2002), and the second forms the outline of the stimulus (Barlow & Reeves, 1979; Carmody et al., 1977; Wenderoth, 1995, 2002). We are less sensitive to information contained between the midline and the outline. It is unclear whether different mechanisms are encoding these two sources of symmetry information, or whether seemingly different mechanisms might just be different expressions of one underlying mechanism. 

More detailed studies have been carried out on the properties of symmetry perception in the central region. The “receptive field” for symmetry in that region has a 2:1 size ratio elongated along the axis of symmetry it encodes (Dakin & Herbert, 1998), is less sensitive to parallel orientations (Dakin & Hess, 1997; Rainville & Kingdom, 2000), and is selective for dot density (Rainville & Kingdom, 2002; but see Dakin & Herbert, 1998). Symmetry in that region is systematically biased toward axes at given orientations (Wenderoth, 2002). Symmetry detection can be equated in peripheral vision if stimuli are increased at the same rate as would be needed for positional tasks (Sally & Gurnsey, 2001). Peripheral symmetry detection is less robust to removal of the central region of the symmetry patch, either because it has a smaller integration window (Tyler, 1999), or because it is less robust to information loss (Poirier & Gurnsey, 2005). In this region, symmetry may be encoded as the coalignment of blobs after filtering with orthogonal orientations (e.g., Dakin & Hess, 1997), or with more specialized symmetry mechanisms (see above). 

Less is known about the properties of symmetry detection based on the outline. Unlike central information, symmetry information present in the outline is independent of the symmetry axis' orientation (Wenderoth, 2002). Shape symmetry can be detected peripherally (Julesz, 1971; Wilson & Wilkinson, 2002). Similarly, symmetry perception in textured patches benefit from outline information (Gurnsey et al., 1998; Labonté et al., 1995). Shape symmetry may be evaluated by point-by-point comparisons of salient features, such as corners or convexities (Wilson & Wilkinson, 2002), which have been shown to play a more important role in object perception than edges (Attneave, 1954; Bertamini, 2001, 2004; Biederman, 1987; Habak, Wilkinson, Zakher, & Wilson, 2004; Loffler, Wilson, & Wilkinson, 2003; Poirier & Wilson, 2007; Shevelev, Kamenkovich, & Sharaev, 2003; but see Hess, Wang, & Dakin, 1999; Mullen & Beaudot, 2002). 

Shape symmetry in the brain

Human object perception is achieved via a hierarchy of processing stages: retinal inputs, simple line and edge detectors in V1 (DeValois & DeValois, 1988; Graham, 1989; Hubel & Wiesel, 1968; Wilson, 1991), curvature detectors in V1 or V2 (Dobbins, Zucker, & Cynader, 1987, 1989; Koenderink & Richards, 1988; Wilson, 1985; Wilson & Richards, 1989), intermediate processing of part and shape representations in V4 (Merigan, 1996; Pasupathy & Connor, 2002; van Essen, 1985; Young, 1992), and further object processing in IT and LOC (Desimone, 1991; Gross, 1992; Tanaka, 1996). Results from fMRI studies suggest some extrastriate areas respond preferentially to texture symmetry (Sasaki, Vanduffel, Knutsen, Tyler, & Tootell, 2005; Tyler et al., 2005), which would be consistent with symmetry based on shape. 

Here we extend our biologically plausible model of shape perception (Poirier & Wilson, 2006) such that it can also account for data collected on shape symmetry perception (Wilson & Wilkinson, 2002; see Figure 1 for sample stimuli). This model was used for multiple reasons, including (1) it was built to account for shape perception of stimuli similar to those used in Wilson and Wilkinson (2002), (2) it is biologically plausible, (3) it has built-in invariance to position and a range of sizes, and (4) very little needs to be added or changed to account for symmetry perception. 

Figure 1

 

Stimuli used in Wilson and Wilkinson's (2002) experiment on shape symmetry. Shown in different columns are the different configurations used, and in different rows are the different phase differences. Symmetry is reduced as the different components are increasingly misaligned (i.e., as phase difference is increased), but the increased asymmetry is most easily detectable for certain patterns than others (e.g., “2 + 3” and “face” are easiest to notice, whereas “2 + 7” and “5 + 7” are hardest). Model responses were taken from these stimuli.

Model

The first 4 stages of the model presented here were previously published as a model of shape perception, and readers are referred to the original paper for details (Poirier & Wilson, 2006). Except for one parameter that was allowed to vary (see Fourth stage: Population code for shape section), the model used here is identical to the original publication. Other discrepancies between the two publications reflect removal of model components that are not relevant to the present issues and that have no consequences on the model's performance. Here, we present only an overview of the model's main features relevant for symmetry perception in shapes, as well as a detailed account of the additional 5th stage that encodes symmetry. 

Readers are referred to the Discussion section, where we provide a thorough treatment of several topics relevant to the current model, including (1) the robustness of the current model with regards to parameter changes and stimulus changes (see Model parameters section), (2) a comparison with other models of symmetry (see Other models of symmetry section), and (3) hierarchical organization does not imply a precedence order of task performance (see Generalizing the model section). 

Overview of the model

The model recovers shape symmetry in five stages: (1) contour information is recovered using oriented filters, (2) object center is recovered using higher order filters that respond at the center of concentric contours ( Figure 2, left), (3) local curvature signals ( Figure 2, middle) are recovered around the contour using a few curvature mechanisms tuned to different degrees of curvature ( Figure 2, right), (4) shape is represented as curvature signal strength as a function of orientation around the object's center, and (5) this object-centric curvature information is used to evaluate symmetry ( Figure 4). 

Figure 2

 

Overview of the shape perception model used to derive a population code of shape (Poirier & Wilson, 2006). (Left) Filters used in the computation of object center. Small-scale oriented filters encode the contour, and orthogonal large-scale filters positioned on either side from the center encode occurrences of concentric line elements. (Middle) Filters used in the computation of local curvature. Maximum response occurs when the contour passes through three oriented filters that are combined (multiplication shown here). (Right) Curvature information is relative to object center: curvature mechanisms scale with distance from object center, and they are oriented to prefer accentuated convexities relative to the curvature expected for a circle. Refer to the text and subsequent figures for the dimensions of the filters used.

Stimuli

Closed contours are created using radial frequency (RF) patterns, where the radius of a circle is varied as a function of polar angle ( θ) using a sum of sinusoid functions of various amplitudes, phases, and frequencies (Wilkinson, Wilson, & Habak, 1998; Figures 1 and 3A): 

$R\left(\theta \right)=R_0*(1+\sum _{n=1}^m{A}_n\cos ({\omega }_n\theta +{\varphi }_n)),$

(1)

where R₀ is the mean radius, and ω_n, A_n, and ϕ_n are the frequency, amplitude, and phase, respectively, for each radial modulation (n of m) added into the circle. RF patterns are useful to study intermediate-level shape perception because they provide controls on shape, show global shape processing properties (Hess et al., 1999; Jeffrey, Wang, & Birch, 2002; Loffler et al., 2003; Wilkinson et al., 1998), and are easily modified to create natural shapes such as faces (e.g., Wilson, Loffler, & Wilkinson, 2002; Wilson & Wilkinson, 2002). In the context of symmetry perception, phase alignment of the different components was manipulated to vary the degree of symmetry in the patterns (see Figure 1). Other stimulus parameters including the contour's luminance profile were matched to experimental conditions (see Wilson & Wilkinson, 2002). 

Figure 3

 

Sample filters and responses of the model at various stages of processing. (A) Sample radial frequency contours are convolved with (B) oriented filters for each of 8 evenly spaced orientations, (C) the output of which is thresholded. For each orientation, the output is convolved with (D) a pair of filters oriented orthogonally to first stage filters, and offset along their axis of orientation, the half-wave rectified output of which (E) is summed over the 8 orientations and (F) thresholded to recover the contour's center, which is estimated as the maximum of that 2D distribution. (G) Curvature mechanisms sample the responses of oriented filters at 5 locations (5 curvature mechanisms shown, each has 3 white and 2 black dots; 3 curvature mechanisms are shown with curved lines to emphasize samples belonging to the same mechanism), the overall scale of the curvature mechanism increases with distance from the object center and is oriented to produce preferential responding to peak curvatures pointing away from the center. After (H) multiplication and inhibition of the samples, (I) the curvature mechanisms respond preferentially at the location of convex peaks. The response of curvature mechanisms (I) is sampled using 2D Gaussian profiles (K; 30 samples shown overlapped). Curvature response is thus recovered as a function of orientation around the center (J, L).

First stage: Oriented filters

Stimuli were convolved with a bank of 8 oriented filters (Wilson, 1985; Figure 3B) distributed evenly over 180°, with parameters adjusted to mimic 8 cpd selectivity (Wilson, McFarlane, & Phillips, 1983). A broader orientation selectivity (i.e., length set to 1/2 of the estimate of Phillips and Wilson, 1984) was used to reduce the number of orientations needed in the simulation to obtain a smooth response. Responses were thresholded at 1/3 of the maximum response over orientations and positions (Figure 3C), which removed sideband responses and reduced positional and response noise of curvature mechanisms. 

Second stage: Object position

Object position is needed to build position and size invariance (see Third stage: Curvature detectors section). Object position was recovered using higher order filters that have been shown to recover object center in concentric Glass patterns (Wilson & Wilkinson, 1998; Wilson, Wilkinson, & Asaad, 1997) and radial frequency patterns (Poirier & Wilson, 2006). Responses at each first stage filter orientation were convolved with a filter defined as a pair of difference of Gaussians (DOGs) oriented perpendicular to the contour, each multiplied by a Gaussian radially, and each displaced by ±y_o from the filter's receptive field center (see Figures 2, left and 3D–3F): 

$W(x,y)=(3e^{-x^2/{0.4}^2}-e^{x^2/{1.2}^2})*(e^{-(y+y_0{)}^2/{0.68}^2}+e^{-(y-y_0{)}^2/{0.68}^2}),$

(2)

where the terms contained in the first set of parentheses reflect the profile of the two DOGs, and the two terms contained in the second set of parentheses reflect the positions of the two DOGs. 

This second stage filter was four times larger in each dimension (and four times lower in spatial frequency) than the first stage filter. Orienting this second layer filter perpendicular to the first layer filter orientation results in extraction of contour curvature (Wilson, 1999), thus generating a strong response to the center of concentric shapes (Wilson et al., 1997). The object center was defined as the location of the peak neural response. 

Third stage: Curvature detectors

Curvature mechanisms operate in parallel over the visual field, encoding curvature for all orientations and a range of curvature amplitudes. For the purpose of providing a plausible yet parsimonious model of radial frequency pattern perception, we restricted our analysis to opponent curvature mechanisms optimized to encode deviations from circular shapes. Moreover, unpublished fMRI data from our laboratory support the presence of neurons responding optimally to deviations from circles in the human ventral pathway (Rainville, Yurganov, & Wilson, 2005). 

Curvature mechanisms were modeled as the combination of the responses of several oriented filters arranged along a curved line, where the response is optimal if the contour matches the sampled locations and orientations of the filters ( Figure 2, middle). For example, the curvature responses to a circle are optimal when oriented filters are positioned around the center of the circle, with distance equal to the radius and orientation perpendicular to the radius ( Figures 2, middle and 3G). Oriented filter activity was sampled (summed) within a small positional blur, increasing bandwidth over curvature, position, and orientation. 

Samples were positioned relative to the object center defined in polar coordinates (where (0, 0) is the object center and ( R _center, θ) is the receptive field's output), taking one sample at the receptive field's center ( R _center, θ), two samples at ±Δ θ of the first sample and slightly further away from the center of the object than the first sample ( R _out, θ ± Δ θ), and the last two samples are also positioned at ±Δ θ but placed slightly inward ( R _in, θ ± Δ θ). The radii of the inward and outward samples were scaled with distance from the object center, thereby providing size constancy over a larger range of object sizes. 

Combining sampled contour responses ( S) from center ( R _center, θ) and inward samples ( R _in, θ ± Δ θ) gives peak responses for accentuated convex curvatures (see Figures 3J and 3L), whereas combining responses from center ( R _center, θ) and outward samples ( R _out, θ ± Δ θ) gives a peak response for straighter or even concave deviations from a circle. Sampled contour responses were combined multiplicatively as  

$C{R}_{R_{\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}},\theta }=\sqrt{S_{R_{\mathrm{i}\mathrm{n}},\theta +\Delta \theta }*S_{R_{\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}},\theta }*S_{R_{\mathrm{i}\mathrm{n}},\theta -\Delta \theta }}-\sqrt{S_{R_{\mathrm{o}\mathrm{u}\mathrm{t}},\theta +\Delta \theta }*S_{R_{\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}},\theta }*S_{R_{\mathrm{o}\mathrm{u}\mathrm{t}},\theta -\Delta \theta }},$

(3)

followed by half-wave rectification, such that curvature mechanisms respond preferentially at points of convex curvature extrema. 

Fourth stage: Population code for shape

We now have curvature responses that are selective for points of maximum convex curvature (see Third stage: Curvature detectors section), located around the previously determined center of the object (see Second stage: Object position section), which gives a maximum response at each peak of a radial frequency pattern. Curvature responses were sampled using 30 2D Gaussians positioned around the object center in 12° steps, with their axes of elongation aligned radially (see Figure 3K), giving 30 object-relative curvature responses ( R _curv). 

Finally, the sampled responses were subjected to a Naka–Rushton non-linearity to simulate cell firing rates (Naka & Rushton, 1966), which provides an accurate description of neural firing rates (Albrecht & Hamilton, 1982; Sclar, Maunsell, & Lennie, 1990): 

$R_{\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{l}}=100*R_{\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{v}}^{N\gamma }/(R_{50\%}^N+R_{\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{v}}^{N\gamma }),$

(4)

where R_50% determines the point of the function where cell firing (R_cell) is half of its maximum (100%), N determines the steepness of the function, and γ is an exponential non-linearity applied to curvature responses. These 30 firing rates, one per sampled direction from the object center, give a population code for internal representation of object shape (see Figures 3J and 3L). 

Fifth stage: Symmetry response

Figures 4 and 5 show how symmetry is computed from the 30 firing rates from the fourth stage. First, these firing rates are normalized such that the sum of R _norm equals unity ( Equation 5):  

$R_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\left(\theta \right)=R_{cell}\left(\theta \right)/\sum _{\theta }{R}_{cell}\left(\theta \right).$

(5)

 

Figure 4

 

Diagram of the symmetry mechanism. Curvature mechanism responses are given as a function of orientation around the object's center (see Figures 2 and 3), which are first normalized to sum to unity. Then, local symmetry is implemented on each pair of normalized responses, such that the sum of responses is inhibited by an amount proportional to its absolute difference. That is, the sum is unchanged if both responses are equal, and reduced otherwise. Finally, global symmetry equals the sum of local symmetries consistent with a given symmetry axis orientation and equals unity for perfectly symmetrical patterns and reduces otherwise.

Figure 5

 

Sample computations of the symmetry mechanism for three patterns, namely: “2 + 3”, “2 + 7”, and “Face”. Columns are divided in two sections, i.e., for stimuli that are symmetrical or not (i.e., phase difference of 30°). For stimuli that are not symmetric, curvature responses are shown for the object (solid line) and its mirror image (dotted line), as well as some local computations of the symmetry model (i.e., solid line shows average response, dotted line shows difference, and dashed line shows curvature response after inhibition). Note that larger signals are more susceptible to inhibition when asymmetries occur. All axes are in arbitrary but common units. Global symmetry measures are 73.75%, 80.95%, and 75.75% for the “2 + 3”, “2 + 7”, and “face” stimuli, respectively.

Then, for each pair of signals, symmetry is defined using shunting inhibition (Michaelis & Menten, 1913): 

$R_{\mathrm{s}\mathrm{y}\mathrm{m}}(\theta ,\varphi )=\frac{0.5\left(R_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\right(\varphi +\theta )+R_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}(\varphi -\theta \left)\right)}{1+{\omega }_{\mathrm{i}\mathrm{n}\mathrm{h}}\left|R_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\right(\varphi +\theta )-R_{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}(\varphi -\theta \left)\right|},$

(6)

where ϕ is a given possible axis of symmetry, θ is the angle around the object center from that axis of symmetry, and ω_inh is a weight regulating the mutual inhibition due to deviations from symmetry. R_sym equals R_norm for symmetrical patterns and decreases monotonically as patterns become increasingly asymmetrical (i.e., as the absolute difference between R_norm · (ϕ + θ) and R_norm · (ϕ − θ) increases). Finally, the degree of perceived symmetry for the pattern (S_pat) for a potential axis of symmetry (ϕ) is given by summing over orientations: 

$S_{\mathrm{p}\mathrm{a}\mathrm{t}}\left(\varphi \right)=\sum _{\theta }{R}_{\mathrm{s}\mathrm{y}\mathrm{m}}(\theta ,\varphi ),$

(7)

and an axis of symmetry will be perceived whenever S_pat peaks, with symmetry strength proportional to S_pat, and the axis orientation equal to the ϕ value where the peak occurred. This not only allows the derivation of the axis of symmetry's orientation and strength but also allows for multiple peaks to occur. That is, the model recovers multiple axes of symmetry when appropriate. Because of the normalization phase (see Equation 5), S_pat equals unity for symmetrical patterns and decreases monotonically as symmetry is decreased. 

Parameters

In the original experiment (e.g., Wilson & Wilkinson, 2002), the axis of symmetry was jittered within ±5° around the vertical axis, thus participants knew to search for near-vertical symmetry axes. That is, participants were likely to ignore off-axis symmetry, such as occasionally occurs when stimuli have phase-misaligned high-RF components. Similarly, we biased perceived axes of symmetry of the model toward the vertical by multiplying S_pat with a Gaussian function (σ = 120°; the specific function used here or its width is not critical to the results), effectively removing these “accidental” axes of symmetry from influencing performance. It is currently unknown what the responses of participants would be if the symmetry axis' orientation was random, if participants' responses would be affected by off-axis symmetry, and whether simply removing or adjusting this bias would be sufficient to account for data from such a scenario. 

We sampled S _pat values in steps of 15° phase shift and interpolated for intermediate values. We used a threshold S _pat value of 0.8 to determine what participants would classify as symmetrical or not; this parameter was not considered a free variable, as it covaries with ω _inh without significantly improving the quality of the fit. The free parameters N and ω _inh were adjusted to decrease the sum of squared differences between the human and model data, using the fminsearch function in Matlab (Mathworks). All other parameters are taken from the low-RF mechanism (see Poirier & Wilson, 2006), which was the mechanism responsible for detection and discrimination performance for the lower range of radial frequencies, as well as lateral masking data collected on RF5 patterns. 

Natural faces

Several changes were made to the model to enable it to recover symmetry in natural faces: (1) edge coding used odd-symmetric oriented filters at a lower spatial frequency (Gabor filters, sine spatial frequency = 2.4 cpd, Gaussian σ = 0.15°), which is better-suited to natural edges, (2) the object center was defined as the weighted average position of the second stage filters, (3) curvature responses were computed within a range of ±23° of the concentric orientation in 5 equal steps, and the maximum curvature response over that range was taken, to increase the curvature mechanisms' orientation bandwidth. 

Results

Model outputs

Figure 6 shows model outputs for some sample stimuli, including a simple geometric shape, synthetic faces, natural faces, and random-dot symmetrical patterns. In all of these cases, the object center was correctly identified. Moreover, from the population code of shape, the correct axes of symmetry were identified for simple geometric shapes and front-view synthetic faces, identical to those we would identify as the axes of symmetry. The model generalized well to natural faces and random-dot symmetrical patterns, and the correct axis of symmetry was recovered for every pattern shown. In the case of natural and synthetic faces, there was some range of symmetry axis orientations over which the symmetry signal remained high. With random-dot symmetry, the symmetry signal dropped rapidly away from the axis of symmetry. 

Figure 6

 

Model performance on a simple shape, on synthetic faces, on natural faces, and on two random-dot symmetrical patterns. Stimuli (row 1) are filtered to extract contours and object center (row 2; edges in white, center in black). Then, a population code of shape is built based on object-centric curvature signals (row 3), from which the model extracts axes of symmetry (row 4; lighter means more symmetrical). The model recovers the correct axes of symmetry for simple shapes and front-view faces but is biased toward the axis of elongation for other views of faces. This reflects a true shift of the symmetry axis rather than a preference for elongation (see Wilson & Wilkinson, 2002, for more details). In addition, the model extracts the correct symmetry axis in random-dot stimuli. See text for details.

Symmetry thresholds

Participants in Wilson and Wilkinson's (2002) experiment had to discriminate between symmetric and asymmetry shapes, and report which of two consecutively presented shapes was the asymmetrical one. These shapes were defined as the combination of two or more radial frequency components into one contour. Those radial frequency components were aligned to create symmetrical shapes, with increased phase difference between the components creating increasingly asymmetrical shapes. For each participant, threshold phase difference of the components was measured for a variety of combinations of radial frequencies (see Figure 1). Acuity to changes in symmetry was dependent on the specific combinations of radial frequencies used, as shown here in Figure 7 (right). 

Figure 7

 

(Left) Model symmetry signal as a function of phase difference, for the different patterns used in Wilson and Wilkinson's (2002) experiment. The thick dashed line at 0.8 shows the threshold value. (Right) Threshold phase differences for model (y-axis) and humans (x-axis).

From the simulations, it is clear that symmetry drops as a function of phase difference for all stimuli (see Figure 7, left). More interestingly, symmetry drops faster for some stimuli, indicating that asymmetries will be easier to detect in those stimuli. Threshold phase for each stimulus was set as the point where the curve crossed the threshold symmetry value of 0.8. The correlation between predicted threshold and human thresholds is high ( Figure 7, right; r(5) = 0.904, t = 4.73, p = 0.0052; or assuming slope = 1 and intercept = 0, r(7) = 0.900, t = 5.16, p = 0.0013), indicating that the model replicates human performance in this task. Replacing our neural implementation of symmetry (i.e., Equations 5– 7) by the more common but biologically implausible cross-correlation operation does not alter the quality of the fit. 

Discussion

We present here the first model of shape symmetry that is based on an existing and biologically plausible model of shape perception. The model predicts perceptual data for a variety of shape-related tasks (Poirier & Wilson, 2006), which now includes shape symmetry perception (present study). Moreover, with small modifications to edge encoding, the model can extract symmetry in natural faces, which would be problematic for conventional symmetry models. These points are discussed in more details below. 

Model parameters

The current model relies on opponent curvature filters. The choice of opponent-curvature mechanisms rather than simple curvature mechanisms or edge representations is warranted to avoid spurious responses that could introduce noise into the signal. The reduction of spurious responses is increasingly important as the stimulus becomes more naturalistic and more complex, i.e., includes other edges or textures. For example, had we used oriented edge operators instead of curvature mechanisms, a radial frequency pattern would produce an oriented-edge response at all points around the shape, thus making it difficult to encode symmetry. Similarly, opponency restricts the analysis to convex curvature, which is likely to exclude from the analysis any segment that is either concave or ambiguous with regards to direction. However, for stimuli where such ambiguity is removed such as the artificial faces, we do not expect opponent curvature to improve the model's performance relative to simple curvature mechanisms. 

The model presented here should hold very well to changes in shape size. First, we showed in the original paper that the model follows human size invariance over a 3.5-fold size change. In addition, the normalization step ( Equation 5) assures that even in the presence of size-related changes in the shape signal used by the symmetry estimation mechanism, the symmetry computation itself would be quite invariant with respect to size. That is, our model will encode symmetry for shapes of different sizes at least as well as it can encode the shape itself. It can also encode various degrees of convex curvatures, although the model would be sensitive to differences in curvature amplitude. Indeed, the model would be less sensitive overall to differences in distance between the stimulus center and the points of maximum curvature, provided it does not alter the recovered object position. The model should be more sensitive overall to changes in curvature amplitude and also to the angular arrangement of curvature maximums. This is because the model uses the curvature's angular position and amplitude to compute symmetry, but not so much its distance to the object center. 

Figure 8 shows the response non-linearity applied to get the population code of shape, both for the original paper (i.e., Poirier & Wilson, 2006; N = 1.942) and the present study (Equation 4; N = 0.288). The remaining free parameter ω_inh was 163.7. Also plotted on Figure 8 are the relevant curvature response ranges for different data sets that were included in the fits. The new value of N creates a more gradual response increase, producing a more continuous code for shape, whereas the earlier parameter produced a sharper increase, which is perhaps more important for threshold detection of the patterns. Clearly, if the previous parameters were kept as is in the present simulations, the symmetry response would be dominated by the rare cases where curvature mechanisms produced low responses. In addition, as noted in the original paper (see Poirier & Wilson, 2006), the model is robust over a 10% change in many of its parameters, N included. A new parameter fit confirms that this robustness could be extended to encompass the current value of N. This parameter fit used the value of N used in this study, and only varied the parameters that were not critical for the present study (e.g., parameters regulating masking or percent correct in identification tasks). The resulting quality of fits was within the range of fit qualities shown in Poirier and Wilson (2006) when parameters were changed by 10%. Thus the model is robust when the value of N is changed to the current value. 

Figure 8

 

Comparison of compressive non-linearity parameters (slope and midpoint) from the previous publication (solid thick line) with those found to fit the symmetry data (dotted thick line). Also plotted are curvature mechanisms' response statistics for three data sets (i.e., circles, detection patterns for radial frequencies 3–6, and symmetry patterns; circles from left to right show the minimum, log-average, average, and maximum, respectively for each data set; note that the range shown for the circle represents simulation noise; data ranges are placed such that the average falls on the compressive non-linearity used in Poirier & Wilson, 2006). Clearly, the compressive non-linearity used previously appropriately increases sensitivity around threshold but compresses most of the relevant variance in the symmetry responses such that sensitivity is effectively decreased. See text for details.

Other models of symmetry

Most modeling efforts in symmetry perception have focused on texture symmetry, with stimuli consisting of randomly placed dots, random checkerboard patterns, or filtered noise. Except for differences in the details of the computations, these models have in common that local low-level features of the image are matched, rather than higher order features. The inputs to the symmetry mechanism would come from V1/V2 cells. An implicit assumption, therefore, is that these mechanisms operate prior to much of the machinery that deals with occlusion, edge assignment, object localization, light source, shape-from-shading, or viewpoint variance. That mechanism's ability to encode symmetry would be seriously compromised by introducing such manipulations. This deficit can be alleviated by additional higher level symmetry mechanisms that compensate for situations that lower level symmetry mechanism cannot encode properly, and/or feedback from object-processing mechanisms onto the inputs to the lower level symmetry mechanisms. 

Another proposal for object shape perception and symmetry has been made (Kurbat, 2002) based on the symmetric axis transform (Blum, 1973). This model is similar in principle with grassfire or erosion models, in that the shape is successively trimmed until only a stick-figure representation remains. Alternatively, object parts could be coded separately using their centroids and axes of elongations. Either way, although such a reduction in the amount of information that enters a shape symmetry algorithm is useful in explaining the efficiency observed in symmetry perception, it is not clear how it could be used to account for psychophysical data on shape symmetry. Specifically, the steps required to generate thresholds have to be made explicit. 

Another model of shape symmetry involves computing the effort needed to morph a shape into its nearest symmetrical (Zabrowsky & Algom, 2002). That is, the distance is calculated between each point of the asymmetric object and its corresponding point on the symmetric object most similar to it. This computation is performed after normalization for size, thus providing size invariance. They show that for a set of random shapes, perceived “goodness” of shape (defining goodness was left up to the participants) was well correlated with a combination of reflexive symmetry and rotational symmetry. It is not clear however what would be the neural implementation of such a model, or how well thresholds derived using it would be correlated with psychophysical thresholds (e.g., Wilson & Wilkinson, 2002). 

Another model of symmetry uses the “weight of evidence” (WoE) computation to account for novel effects relating to symmetry-to-noise ratios (Csathó, van der Vloed, & van der Helm, 2004). The WoE computation is simply WoE = S/n, where S is the number of symmetry pairs, and n is the total number of elements. As the bulk of their argument relates to this computation, it is worthwhile comparing with our own computation. Although the details of the implementation differ, our model also does a computation that is conceptually similar to W = S/n. The division by n is done in our model by normalization (Equation 5), and the sum of S is done by adding (Equation 7) the normalized responses after shunting inhibition removes the responses that are asymmetrical (Equation 6). Thus, Equations 5–7 perform a computation equivalent to the WoE implementation. It follows then that any conclusion derived from the WoE will equally apply to our model. The only notable differences are (1) we use shunting inhibition to classify whether responses are random elements or pairs, and this classification varies gradually with the difference between signals of a pair, (2) the symmetry computation is done on a different internal representation of the stimulus, and (3) the two models were developed to account for performance with regards to different classes of stimuli. 

A recent model by Dry (2008) describes the stimulus using Voronoi tessellation. Voronoi tessellation is in many ways similar to grassfire models presented above, in that for each point, it defines a region that contains every position that is closest to that point. However, instead of using these regions to reduce stimulus complexity, Dry's model compares whether the same region rotated to the opposite side of the symmetry axis also contains only one dot. If symmetrically placed regions do not each contain exactly one dot, then the stimulus is perceived as less symmetrical. Thus its operation is similar to a pointwise comparison, with some resistance to jitter proportional to the inverse of local dot density. That is, as local dot density increases, the model tolerates less positional jitter. It provides a good fit to empirical data. However, it is not clear what would be the neural implementation of such a model. It is also unclear how the model would need to be adapted to encode symmetry in more natural stimuli like shapes or faces. 

Figure 6 shows that the current model can extract the symmetry axis from random-dot patterns, and this even without the adjustment of the parameters used for natural images. However, much research is needed before any claim is made to relate our model to how humans extract symmetry from such patterns. Indeed, the model was created to extract symmetry from roughly circular shapes, and in both random patterns it extracts hints of a roughly circular shape. This emphasis is different from the usual proposal that information near the symmetry axis is more important to the task. We can speculate that this model, if involved in symmetry perception in random-dot patterns, is most likely more involved in recovering symmetry of the outline or of a shape contained within the pattern. As such, our model could be complementary to a model recovering symmetry along the midline, both operating on similar principles. 

Generalizing the model

The model presented here accounts for radial frequency (RF) pattern perception in a variety of tasks (Poirier & Wilson, 2006) including symmetry (present study). We have shown here with several examples that when the orientation filter's spatial frequency selectivity matches that of the contour, the present model robustly recovers the axes of symmetry. Evidently, model performance would improve further if other mechanisms were in place to extract a more reliable and more precise contour signal. Moreover, the shape signal would be less noisy if the outputs of several curvature mechanisms were combined, each tuned to different degrees of curvature, different orientations, and different scales, rather than the single mechanism used here. The neural representation of objects may also include information about curvature distance from the object's center, in addition to radial position and curvature amplitude used here. Object shape would then be a population code of curvature responses, mapped as a function of distance and orientation around the object's center, as proposed by Pasupathy and Connor (2002). 

The model presented, despite its hierarchical organization, is consistent with both local-to-global and global-to-local processing schemes (see Chen, 2005), where a global-to-local processing precedence are evident when information at later hierarchical stages is available earlier to consciousness than information encoded at earlier hierarchical stages (e.g., Ahissar & Hochstein, 2004). Moreover, the performance of the current model could be improved significantly by additional feedback or closed-loop processing. Global information could also be privileged if asymmetries exist between integration and segregation (see Poirier & Frost, 2005) and between low-level and high-level information (see Poirier & Wilson, 2007). 

Future directions

The modeling here provide some novel answers regarding shape processing and symmetry in human vision and is flexible enough to adapt to a range of stimuli such as synthetic and natural faces. One main contribution is the proposal of simple ways in which invariance may be achieved. The original model (Poirier & Wilson, 2006) incorporated invariance to object position and size over a given range. Conversely, the extraction of a symmetry axis (or axes) can help in building orientation invariance of shape processing by providing a common frame of reference for object processing in addition to elongation and other cues (e.g., Figure 9). As more elements of the puzzle are put together, it is obvious that system synergies occur that provide a much more robust response than would be expected from the components. 

Figure 9

 

Interactions of facial features and head shape in determining symmetry. We modeled shape symmetry above, but it is possible that a similar feature-based symmetry mechanism would be used to encode symmetry of facial features. The mechanism used to find head location in the image could be used to constrain the search of symmetry axes positions, and the symmetry axis found based on shape information could constrain further the orientation of possible feature-based symmetry axes. Estimates of symmetry axes based on head shape and on internal features could then be combined.

An issue that proved critical to the success of the mechanisms across stimuli (i.e., in natural and synthetic images) was whether the receptive fields encoding the contours were appropriate for the stimuli. Variability in responses at the oriented receptive field level, whether they arise from image contrast differences or mismatches in spatial frequency in the image and in the receptive fields, has multiplicative effects at the level of curvature mechanisms and above. Simple modifications of spatial-frequency selectivity either by combining responses across spatial-frequency channels or selecting a spatial frequency more appropriate for the stimulus considerably reduced this problem. Clearly, the present model needs to incorporate biologically plausible mechanisms that deal with contrast and spatial-frequency invariance and can account for effects of contrast modulations and occlusions (see Habak et al., 2004; Hess et al., 1999; Poirier & Wilson, 2007). 

Finally, it is still unclear whether curvature scaling arises from top-down or bottom-up mechanisms. Top-down mechanisms would select the mechanisms appropriate for the scale of processing involved. Bottom-up selection would occur as a consequence of increasing invariance over spatial frequency, scale, and curvature, where the mechanisms involved are selected simply because they are the most sensitive to the shape differences at that scale and spatial frequency. 

Acknowledgments

This research was supported by an NSERC Operating Grant to HRW (#OP227224). Portions of this paper were presented at VSS 2006 (Sarasota, Florida). 

Commercial relationships: none. 

Corresponding author: Frédéric J. A. M. Poirier. 

Email: [email protected]. 

Address: Laboratoire de Psychophysique et Perception Visuelle, Université de Montréal, 3744 rue Jean-Brillant, Bureau 210-46, Montréal, Québec H3T 1P1, Canada.