By utilizing statistical properties and summary statistics, the visual system can efficiently integrate perception of spatially and temporally adjacent stimuli into perception of a given target. For instance, perception of a target face can either be biased positively toward previous faces (e.g. the serial dependence effect) or be biased negatively by surrounding faces in the same trial/space (e.g. spatial ensemble averaging). However, both aspects were investigated separately. As spatial and temporal processing share the same purpose to reduce redundancy in visual processing, if one statistical processing occurs, would the statistical processing in the other domain still exist or be discarded? We investigated this question by exploring whether serial dependence of face perception (of attractiveness and averageness) survives when the changed face perception in the group context occurs. The results of Markov Chain modeling and conventional methods suggested that serial dependence (the temporal aspect) co-occurs with changed face perception in the group context (the spatial aspect). We also utilized the Hidden Markov modeling, as a new mathematical method, to model statistical processing from both domains. The results confirmed the co-occurrence of temporal effect and changed face perception in the group context for both attractiveness and averageness, suggesting potentially different spatial and temporal compression mechanisms in high-level vision. Further modeling and cluster analysis further revealed that the detailed computation of spatially and temporally adjacent faces in the attractiveness and averageness processing were similar yet different among different individuals. This work builds a bridge to understanding mathematical principles underlying changed face perception in the group context from the serial perspective.

*Generator*creates “fake” images, and the

*Discriminator*estimates the probability that an incoming image comes from the training set rather than from the

*Generator*. Competitions between the

*Generator*and

*Discriminator*drive the outcome of “fake” facial images indistinguishable from authenticated images. The stimuli were from a database which uses the StyleGAN framework (http://seeprettyface.com). Out of the 50,000 computer-generated faces from the database, we manually selected the 150 female faces (see a sample in Figure 2A) as stimuli used in the present study, which were: (1) Eastern Asian, (2) with a wide range of facial attractiveness, (3) with neutral facial expression, and (4) with frontal viewpoint and in a direct gaze direction. The faces were all presented in grayscale and were carefully aligned and trimmed via Photoshop (Adobe Inc.) at 200 × 200 pixels.

*K*-means cluster analysis to further model the emission probability matrix of Hidden Markov models, and test whether the participants have one unified or several different pattern(s) of the contrast effect across time.

*Y*

_{1},

*Y*

_{2}, ...,

*Y*) are a Markov chain with values in the hidden state space {1, 2, ...,

_{N}*k*}, where

*k*represents the number of the hidden states (Pardoux, 2010). The Markov Chain of the latent variables is defined by two unknown parameters (

*μ*,

*P*), where

*μ*represents initial state probability, and

*P*is transitional probabilities between the latent variables (Pardoux, 2010). Under the condition of (

*Y*

_{1},

*Y*

_{2}, ...,

*Y*) = (

_{N}*y*

_{1},

*y*

_{2}, ...,

*y*

_{n}), the observed sequence (

*X*

_{1}, ...,

*X*

_{n}) is independent. The distribution of each

*X*is dependent only on

_{k}*y*, and is the given function of yk. In other words, for any 1 ≤

_{k}*n*≤

*N*,

*function from the Statistics and Machine Learning Toolbox of MATLAB.*

**hmmestimate***K*-means clustering process (Hartigan & Wong, 1978).

*rho*= 0.86,

_{Spearman}*p*< 0.05).

*t*-tests confirmed existence of the changed face perception in the group context at attractiveness (

*M*= 0.21, SEM = 0.015,

*t*(6899) = 14.44,

*p*< 0.001, Cohen's

*d*= 0.17), and averageness (

*M*= 0.08, SEM = 0.018,

*t*(6899) = 4.60,

*p*< 0.001, Cohen's

*d*= 0.06). The repeated measures correlation analysis (Figure 3; Bakdash & Marusich, 2017; Burns et al., 2021; Ying et al., 2019) suggested that the difference between the group mean and the target negatively predicts magnitude of the attractive (

*r*= −0.44,

*p*< 0.001, 95% confidence interval [CI] = −0.46 to −0.42) and averageness friend effect (

*r*= −0.58,

*p*< 0.001, 95% CI = −0.60 to −0.56). To cross-validate the results of the changed face perception in the group context, the linear mixed-effect model was used to investigate the influence of the contrast effect on the target face judgment in attractiveness and averageness, respectively. The results supported the contrast account of the changed face perception in the group context, which is consistent with the previous studies (Burns et al., 2021; Ying et al., 2019; see table S2 in the Supplementary Material).

*r*

_{att}= 0.50,

*p*< 0.01;

*r*

_{ave}= 0.56,

*p*< 0.01), indicating the positive serial effect of the previous target ratings on the current target ratings. To cross-validate the Markov Chain modeling, the linear mixed-effect model was also used to investigate serial dependence of facial attractiveness and averageness judgments, which supported the current results (see Table S1 in the Supplementary Material).

*r*= 0.80,

*p*< 0.01), indicating the generally common processing pattern of the changed face perception in the group context shared by attractiveness and averageness.

*r*= 0.33,

*p*= 0.21; averageness:

*r*= 0.44,

*p*= 0.09). These findings suggested that: (1) there exists the contrast effect in attractiveness and averageness, but (2) these contrast effect(s) are not simple linearly.

*R*

^{2}

_{DoG}= 0.17 vs.

*R*

^{2}

_{Linear}= 1.55E-15; averageness:

*R*

^{2}

_{DoG}= 0.23 vs.

*R*

^{2}

_{Linear}= 1.06E-14). Thus, the contrast effect may have a nonlinear and negative effect on the changed face perception in group context for both facial traits at certain contrast levels. However, as predicted by the DoG models, the influence of the contrast effect on the face perception change tends to be attenuated at the extreme level of the contrast effect.

*K*-means cluster analysis to test whether there is/are one unified or several different pattern(s) of the contrast effect of individual participants based on the emission probability matrices of the Hidden Markov model (Barber, 2012). Before the

*K*-means cluster analysis, we estimated the optimal number of clusters using the gap statistic (Tibshirani et al., 2001). Gap statistic estimates the fair

*K*number by calculating the gap statistics for each

*K*(Tibshirani et al., 2001). The minimal optical

*K*was decided when the gap statistic of

*K*is larger than the difference between the gap statistics of

*K*+ 1 and its standard error, Gap(

*K*) > Gap(

*K*+ 1) - SEM(K + 1).

*K*= 3 fitted the best, thus the results favored there exists three patterns/clusters of the contrast effect (see Figure 5C). Most participants (at 65%; 15 out of 23) are sorted as pattern 3 (dotted green line): the contrast effect is prominent at the observed-state-3 (being slightly more attractive) for the hidden-state-1 and 2 (less attractive surrounding faces), but at the observed-state-1 (being severely less attractive) for the hidden-state-3 and 4 (more attractive surrounding faces). Fewer people are sorted as the patterns 1 and 2 (both at 18%; both 4 people), clearly different from the pattern 3. However, for averageness, the results suggested there exists only one pattern of the contrast effect in changed perception of facial averageness in the group context (see Figure 5F). The contrast effect is stronger at the hidden-state-1 and 4 and less obvious at the hidden-state-2 and 3. In general, the

*K*-means cluster analyses suggested that the detailed mechanisms of the contrast effect(s) are similar yet different in the attractiveness and averageness friend effect, and the relationships between the contrast effect and the changed face perception in the group context should be nonlinear.

*K*-means cluster analysis to the Hidden Markov models (Hartigan & Wong, 1979). Although the previous study which investigated face perception in the group context using the linear mixed model did not find a significant difference between facial attractiveness and averageness, the cluster analysis on the Hidden Markov models for attractiveness and averageness suggests that the detailed processing mechanisms of the changed face perception in the group context are slightly different. Specifically, the

*K*-means cluster analysis favored one consistent pattern of emission matrices in averageness (see Figure 5F) but unveiled different patterns of emission matrices among participants for attractiveness (see Figure 5C). The cluster analysis applied here is an exploratory analysis of individual differences in face perception in the group context across time, which supports individual differences in processing mechanisms of face perception in the group context for attractiveness but not for averageness. Such a finding from Hidden Markov model follows the existing literature showing the complicated relationship between the two traits. While it is true that averageness (sometimes measured under the term of “distinctiveness”) is a vital part of attractiveness (e.g. Langlois, Roggman, & Musselman, 1994; Little et al., 2011; Perrett et al., 1999), it is well believed that perception of attractiveness is also formed by other physical (e.g. symmetry, skin color; and secondary sexual characteristics) and other factors (e.g. personality, Hormone level, rater's own attractiveness). Here in this study, the exploratory analysis using

*K*-means cluster analysis differentiated the computational mechanisms of attractiveness and averageness: the relatively more complicated trait, attractiveness, can be computed via different mechanisms by different patterns; whereas the relatively less complicated trait, averageness, is unanimously computed. It is very likely that the tentative cluster analysis successfully captured the individual differences when computing these traits. For averageness, as its processing is primarily focused on the physical features of a face, different participants exhibited one pattern of the temporal and spatial integration. For attractiveness, as its processing is more complicated (individuals may incline to physical features disproportionally) and involves personal characteristics of the “rater,” the data suggested that there are (at least) three patterns of processing mechanisms. Therefore, cluster analysis in the Hidden Markov modeling supported that the detailed processing mechanisms of face perception in the group context might be different between attractiveness and averageness (see Figures 5C, F). Therefore, the Hidden Markov modeling has significant advantages over conventional methods. Future studies may employ different stimuli and modeling methods to further investigate the computational differences between these two traits.

*K*-means cluster analysis, data suggested the detailed computations of spatial and temporal adjacent faces in perception of attractiveness and averageness were similar yet different for different individuals. The findings expanded our understanding of serial face perception in the group context and offered a new analysis method to unveil the mathematical principles of the spatial relationship among stimuli from the serial perspective.

*Undergraduate Research Advising Project*.

**Data are available at:**https://osf.io/h9dsj/.

**Authors’ contributions: J.M. Yu:**data analysis, data visualization, and writing.

**W. Yang:**coding and suggestion.

**H. Ying:**conceptualization, coding and design, data analysis, data visualization, funding and resources acquisition, and writing.

*Trends in Cognitive Sciences*, 15(3), 122–131. [CrossRef]

*Frontiers in Psychology*, 8, 456. [CrossRef]

*Bayesian reasoning and machine learning*. Cambridge: Cambridge University Press.

*Bulletin of the American Mathematical Society*, 73(3), 360–363. [CrossRef]

*eLife*, 9, e54172. [CrossRef]

*Scientific Reports*, 7(1), 14739. [CrossRef]

*Cognition*, 212, 104715. [CrossRef]

*Scientific Reports*, 9(1), 9329. [CrossRef]

*British Journal of Psychology*, 112, 902–933. [CrossRef]

*Nature Reviews Psychology*, 1(6), 316–316. [CrossRef]

*Proceedings. Biological Sciences*, 285(1890), 20181722.

*Nature Communications*, 13(1), 5741, https://doi.org/10.1038/s41467-022-33508-1

*Journal of Experimental Psychology: Human Perception and Performance*, 33(6), 1420.

*Nature Neuroscience*, 17(5), 738–743. [CrossRef]

*eLife*, 9, e55389. [CrossRef]

*Journal of Experimental Psychology*, 16(1), 1–31.

*Scientific Reports*, 12, 10746.

*Current Biology: CB*, 17(17), R751–R753, https://doi.org/10.1016/j.cub.2007.06.039

*Applied Statistics*, 1326(28), 100–108.

*Finite Markov chains and algorithmic applications*. Cambridge: Cambridge University Press.

*Perception & Psychophysics*, 3(6), 409–414.

*IEEE Transactions On Pattern Analysis And Machine Intelligence*, 43(12), 4217–4228.

*Psychological Science*, 5(4), 214–220.

*Frontiers in Psychology*, 11, 2258.

*Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences*, 366, 1638–1659.

*Scientific Reports*, 7(1), 1971, https://doi.org/10.1038/s41598-017-02201-5

*Frontiers in Psychology,*12, 607448.

*Nature Neuroscience*, 4(7), 739–744.

*Spatial Vision*, 10(4), 437–442.

*Nature*, 368, 239–242.

*Evolution and Human Behavior*, 20(5), 295–307.

*Perception*, 30, 611–625, doi:10.1068/p3123.

*Psychological Science,*22(9), 1183–1190.

*Psychological Science*, 26(1), 39–47.

*Proceedings of the National Academy of Sciences of the United States of America*, 117(19), 10218–10224.

*Journal of the Royal Statistical Society B,*63(2), 411–423.

*Psychonomic Bulletin & Review*, 11(3), 482–487, https://doi.org/10.3758/BF03196599.

*Journal of Vision*, 19(12), 6.

*Memory & Cognition*, 20(3), 291–302.

*Psychological Science*, 25(1), 230–235, https://doi.org/10.1177/0956797613497969

*Vision Research,*104, 68–79.

*Annual Review of Vision Science,*1, 547–567.

*Annual Review of Psychology,*69, 105–129.

*Journal of Vision*, 16(15), 28, https://doi.org/10.1167/16.15.28

*Journal of Experimental Psychology: General*, 148(3), 421.

*Cognition,*195, 104128.

*Journal of Vision*, 21(13), 4.

*Perception*, 51(4), 276–285.

*Behavior Research Methods*, 50(4), 1496–1502.