Temporal enhancement of cross-adaptation between density and size perception based on the theory of magnitude

Abstract

The ability to estimate spatial extent is an important feature of the visual system. A previous study showed that perceived sizes of stimuli shrank after adaptation to a dense texture and that this density-size aftereffect was modulated by the degree of density. In this study, we found that the aftereffect was also modulated by the temporal density of the adapting texture. The test stimuli were two circles, and the adapting stimulus had a dotted texture. The adapting texture refreshed every 67 to 500 ms, or not at all (static), during the adaptation. The results showed that the aftereffects from a refreshing stimulus were larger than those under the static condition. On the other hand, density adaptation lacked such enhancement. This result indicates that repetitive presentation of an adapting texture enhanced the density-size cross-aftereffect. The fact that density modulation occurs in both the spatial and temporal domains is consistent with the theory of magnitude, which assumes that the processing of the magnitude estimation of space, time, and numbers share a common cortical basis.

Introduction

We can visually estimate distances between trees in a park even though there is nothing in that space. It is strange that our visual system can estimate “vacant space” despite it receiving no signal from that space. Concerning the estimation of vacancy, Hisakata, Nishida, and Johnston (2016) reported a new adaptation effect, called the density-size aftereffect, in which the perceived distance and size shrink after adaptation to a dense texture. They argued that this indicates that density plays a role as a metric for the estimation of the size of a circle and that adaptation to a dense texture reduces the scale, causing the circle to appear smaller in size. In other words, apparent density mirrors the change in the scale of the metric itself. This property of adaptation is different from that of traditional size or spatial frequency adaptation, in which the direction of perceptual change is opposite to that of the “adapting stimulus.” Therefore, we considered that the representation of position and spatial frequency at early visual stages should remain unchanged, and that the representation of density modulates estimations of size. A logical question arising from this is as follows: in which stage does size estimation change? 

Concerning the estimation of space, time, and numerosity, Walsh (2003) proposed the theory of magnitude (TOM), a conceptual framework that assumes that the processing of magnitude estimation of space, time, and numbers share a common cortical basis. Regarding the cortical area related to the TOM, the intraparietal sulcus (IPS) seems to be responsible for magnitude estimation because many physiological studies have shown that the judgment of size or orientation, such as the bisection task, and the estimation of lengths of line, numerosity, and duration all activate this region (e.g. Castaldi, Aagten-Burphy, Tosetti, Burr, & Morrone, 2016; Dormal, Andres, & Pesenti, 2012; Dormal & Pesenti, 2009). Researchers have claimed that the magnitude estimation system in the IPS gathers sensory signals from cortical areas with different modalities and then makes a common representation of their magnitudes for action (Anobile, Arrighi, Castaldi, & Burr, 2021; Anobile, Arrighi, Togoli, & Burr, 2016; Walsh, 2003). It is likely that this common magnitude processing system reads the metric represented by density information in the perceptual system. 

Concerning apparent numerosity, Burr and Ross (2008) reported that the apparent numerosity of texture decreases (increases) after adaptation to a large (small) number, indicating that the system for estimating numbers is adaptable. They manipulated the number of dots in both the adapting stimulus and test stimulus, where the adaptation effect was constant when there were at least 12 dots in the test, and decreased as the number of dots approached the subitizing range. Burr and colleagues proposed that, when the number of objects exceeds the subitizing range, a system to estimate numerosity is engaged, that it is different from the system handling stimulus contrast, size, orientation, color, etc. Their series of studies suggested that an adaptable mechanism estimates the number of texture elements. Recently, Aagten-Murphy and Burr (2016) reported that the “numerosity aftereffect” increased with repetition of the adapting stimulus, but not with an increase in the adaptation duration. Their study showed that numerosity adaptation relied on event-based repetition. Other studies showed that this aftereffect occurs in a higher cortical area, such as the IPS (Anobile, Cicchini et al., 2016; Castaldi et al., 2016; Liu, Zhang, Li, Zhao, & Tang, 2015); therefore, it is likely that a common system for magnitude estimation is adopted in the numerosity aftereffect. 

In this study, we investigated the effects of both the repetition and duration of an adapting stimulus on the density-size aftereffect. If the density-size aftereffect demonstrated in Hisakata et al. (2016) shares common mechanisms with the numerosity aftereffect mentioned above, it should have a temporal property similar to that of the latter. In other words, the repetition, not the duration, of an adapting texture should increase adaptation-induced space compression, as with numerosity adaptation. In Experiment 1, we measured the time course of the density-size aftereffect in a similar way to that used by Aagten-Murphy and Burr (2016) for the numerosity aftereffect. We manipulated both the presented repetition number and duration of the adapting texture and measured the perceived size of an object before and after adaptation, to determine whether the repetition or duration of adaptation enhanced the density-size aftereffect. The results showed that both factors affected the aftereffect, but long exposure times were needed for repetition of the adapting texture to enhance the aftereffect. To investigate temporal repetition and the dissociation between the density-size and density-density aftereffects in greater detail, in Experiment 2, we manipulated only the number of repetitions of the adapting texture to assess the effects on both size and density perceptions. We found a repetition effect on the density-size aftereffect, but not on the density-density aftereffect. We assume that repetition of the adapting texture induced processing adaptation for magnitude in higher stage. 

Experiment 1

Materials and methods

Subjects

Three observers participated in the experiment (one of the authors and two observers naïve to the purpose of the study). They all had normal or corrected-to-normal visual acuity. The study was approved by the Tokyo Institute of Technology Ethics Committee and conformed to guidelines of the Declaration of Helsinki on the use of human observers in research. 

Apparatus

Stimuli were generated on a computer (Apple MacBook Pro 2016) and displayed on a liquid crystal display gaming monitor (EIZO FROIS; 27-inch, 2560 × 1440 pixels, refresh rate of 100 Hz, 2.24 arcmin/pix, mean luminance of 54.5 cd/m², gamma-corrected). The viewing distance was 57.3 cm and the size of the adapting texture was 15 degrees × 15 degrees. 

Stimuli and conditions

The adapting texture was composed of white and black dots 10 pixels in diameter (Figure 1a). The lower-density stimulus comprised nine dots, and the higher density stimulus 144 dots in a 15 degrees × 15 degrees area. The test stimulus was an open circle with a black outline. We used the simple staircase method (one up and one down) to measure the perceived size of the test circle. One experimental session involved one staircase, allowing measurement of the time course of the transition in perceived size. The session consisted of three parts: baseline, adaptation, and post-adaptation (Figure 1b). In the baseline part, two circles were presented for 200 ms in the left and right visual fields; one was the standard stimulus (12.25 deg²), and the other was the comparison stimulus (the size varied according to the observer's previous response). Within 1100 ms after presentation, the observer responded by indicating which circle (left or right) was larger using two buttons. Responding as quickly as possible was not explicitly emphasized. The time provided for a response was fixed for 1100 ms, and the next stimulus was always presented 1300 ms after the offset of test stimuli. Each test stimulus (one trial) was presented 60 times for a total of 90 seconds. During the adaptation period, adapting textures were presented between presentations of the test circles. Textures were presented in both the left and right visual fields, with black frames, to prevent a size aftereffect. One of the textures consisted of 144 dots, and the other of nine dots. Dots in the adapting textures were positioned at points on a square grid with a random displacement of up to 30 arcmin, which was either updated every 300 ms (3.33 Hz), every 100 ms (10 Hz), or not at all (static). There were two adaptation duration conditions (1 and 5 seconds). The adaptation part consisted of 60 trials and took 156 or 396 seconds to complete, depending on the adaptation duration. In the post-adaptation part, only the test stimulus was presented, as in the baseline part. 

Figure 1.

 

(a) Example of an adapting texture (left panel) and the test stimulus (right panel). (b) Sequence for one experimental session. In the baseline and post-adaptation trials, only the test stimulus was presented. In the adaptation trials, an adapting texture was presented between presentations of the test stimulus. The size of the comparison stimulus was manipulated according to the staircase method in a one-up, one-down manner.

Data fitting

During each trial, we recorded the size of the stimulus compared to the standard (12.25 deg²). The data for the first 10 trials were excluded from analysis because these included initial fluctuations. We defined the average value from trials 11 to 60 as the baseline, and this baseline value was subtracted from all analyzed data. After the subtraction, we calculated moving averages with a range of ±22 trials. We fitted the following function to the time-course data with a nonlinear least squares method:  

\begin{equation*}\!\!\!\!\!\!\!\!\!\begin{array}{@{}l@{}} f\left( t \right) = \left\{ \begin{array}{@{}*{1}{c}@{}} {a,\;t\; \le \;90}\\ {{S_{max}}*\exp \left( {1 - \left( {\frac{{ - t}}{{{\tau _{adapt}}}}} \right)} \right) + a,\;90 < t \le adapting\;duration}\\ {{S_{max}}*\exp \left( {\frac{{ - t}}{{{\tau _{post}}}}} \right) + a,\;adapting\;duration < t} \end{array}\right.\end{array}\end{equation*}

where t is the time in seconds, a is the baseline of shrinkage before adaptation, which should be approximately 0, S_max is the maximum value of the effect, τ_adapt is the time taken for the maximum value to be reached, and τ_post is the decay time. We defined the difference between a and S_max as the amplitude, which is plotted in Figure 2. Nonparametric bootstrapping was carried out with 10,000 runs to calculate the 95% confidence intervals of these parameters using the percentile method. 

Figure 2.

 

(a) Averaged time course data of perceived size before, during, and after density adaptation for each observer (each indicated by a pair of letters). Under each refresh condition; light red and blue areas indicate the adaptation duration conditions of 1 and 5 seconds, respectively. Red and blue solid lines indicate functions fitted to the data of the adapting conditions, 1 and 5 seconds, respectively. (b) Estimated amplitude of the density–size aftereffect with a bootstrap simulation with 10,000 runs. The amplitude was estimated using the median of the histogram from non-parametric bootstrapping. The error bars indicate 95% confidence intervals estimated with the percentile bootstrap method. (c) Averaged parameters across the observers with the bootstrap simulation run 10,000 times. The error bars indicate 95% confidence intervals, estimated using the percentile bootstrap method. (d) Averaged time course data of perceived density before, during, and after adaptation for each observer under each refresh condition. (e) Estimated amplitude of density–density aftereffects with a bootstrap simulation with 10,000 runs.

We used the same procedure to measure density adaptation. In this case, only the test stimulus was different from the size trials; texture consisted of white and black dots whose positions were randomly assigned within an area of 15 degrees × 15 degrees. The standard texture contained 49 dots, and the numbers of dots for estimating density varied according to the staircase method. The presentation duration of the test stimulus was 200 ms. 

Results

Figure 2a shows the changes in perceived size before, during, and after texture adaptation under all refresh conditions for each observer. Red and green lines in each panel show the results of the adaptation duration conditions of 1 and 5 seconds, respectively. The perceived size of a circle presented on the adapted side shrank in the adaptation phase under all conditions. Interestingly, the shrinkage reached the maximum value right after the beginning of the adaptation phase. 

To show the time course of the adaptation effect quantitatively, we fitted a function and calculated the amplitudes and rise and decay times. After this fitting, we assessed whether differences were significant based on the overlap between 95% confidence intervals estimated with the percentile bootstrap method. The amplitudes of shrinkage were larger for the 5-second adaptation than for the 1-second one under all refresh rate conditions, showing that the adaptation duration is important for the density-size aftereffect. This result is inconsistent with the temporal characteristics of the numerosity aftereffect shown in Aagten-Murphy and Burr (2016). The amplitude of the aftereffect under the static (no refresh) condition was smaller than those with a refresh rate of once every 100 and 300 ms for both adaptation durations of 1 and 5 seconds (Figure 2b). Their results indicate that the refresh frequency of the adapting texture also affects shrinkage, especially under the 5-second adaptation condition. 

To compare this characteristic of size perception with that of density perception, we conducted the same measurement using a textured pattern instead of a circle as the test stimulus. Here, the pattern consisted of dots in random positions, and the observer judged whether the left or right texture was denser. For density perception, the magnitude of the aftereffect did not differ between adaptation conditions of 1 and 5 seconds; furthermore, it did not seem that refreshing the adapting texture affected the magnitude of the aftereffect (Figures 2d,e). This result suggests that the density–density aftereffect saturates even when the adaptation period is short (1 second), whereas the density-size aftereffect was affected by both the adaptation duration and refresh frequency, indicating that the magnitude of the density-size aftereffect is influenced by temporal adaptation. 

Experiment 2

Experiment 1 revealed that repetition of the adapting texture enhanced the aftereffect only with long-duration exposure, whereas no repetition effect was observed on the density-density aftereffect. In Experiment 2, we aimed to clarify the repetition effect. We used a more conventional experimental design with several temporal frequencies for the adapting texture. To examine the effect of repetition on both the density-size and density-density aftereffects directly, we manipulated the repetition number of the adapting texture. 

Materials and methods

Subjects

Eight observers participated in the experiment (one of the authors and 7 observers naïve to the purpose of the study); informed consent was obtained from all participants. The study was approved by the Tokyo Institute of Technology Ethics Committee and conformed to guidelines of the Declaration of Helsinki for human research. 

Procedure and conditions

The apparatus and stimuli were the same as those used in Experiment 1. The duration of the first adaptation period was 60 seconds with a 5-second top-up adaptation in subsequent trials (Figure 3a). Dots within the adapting texture were positioned at points on a square grid with a random displacement of up to 30 arcmin, which was either updated every 66.67 ms (15 Hz), 100 ms (10 Hz), 250 ms (5 Hz), 300 ms (3 Hz), or not at all (static). The center of the stimulus was located at a visual angle of 9.55 degrees relative to the fixation point. We used the one-up, one-down staircase method to determine the point of subjective equality. The position of the adapting visual field was balanced among sessions. Breaks of at least 5 minutes were included between sessions. There were four sessions under each condition and the magnitude of perceived shrinkage was defined as the average of the last 10 trials. For the density-density adaptation measurement, the procedure was the same, except that the test stimulus was a texture composed of 10-pixel-diameter white and black dots positioned randomly within the adaptation area. 

Figure 3.

 

(a) Stimulus presentation in Experiment 2. (b) Results of the aftereffect for each observer (each indicated by a pair of letters) including the average values. The left and right panels show the change in perceived size and density after adaptation, respectively, as a function of the refresh frequency of the adapting stimulus. Error bars indicate the standard errors of the means.

Results

Figure 3b shows the individual and average temporal frequency of the adapting texture. Temporal refreshing of the adapting texture enhanced the aftereffect on size perception (F(4,28) = 10.07, p < 0.0001, h² = 0.206, and Ryan's method multiple comparison showed 0 Hz < other temporal frequencies); on the other hand, it did not change the aftereffect on density perception except under a refresh frequency of 15 Hz (F(4,28) = 41.29, p = 0.034, h² = 0.099, Ryan's method multiple comparison showed 0 Hz < 15 Hz). As in Experiment 1, we found a difference in the characteristics of temporal enhancement for size and density perceptions after adaptation to a dense texture. 

Discussion

Based on the TOM, we hypothesized that the density-size aftereffect should be affected by temporal modulation of the adapting texture, as with the numerosity aftereffect. Here, we showed that repeated presentation of the adapting texture increased the shrinkage in perceived size after adaptation. We consider this an indirect effect of temporal adaptation at the higher stage of estimating a common “magnitude.” If all magnitudes of different perceptual modalities, such as time, space, and number, are estimated in a common cortical area, it is likely that the temporal characteristics of an adapting stimulus are similar for those modalities. There are two possible explanations for this phenomenon. The first is that a single processing mechanism encoding the number of objects adapts when observing the dense texture, which reduces the perceived object size. The second is that the estimation of object size is affected by the density adaptation during the early/middle stage of visual processing. The first possibility suggests that a single processing mechanism for changing magnitude is adapted to process the numerosity of the dense texture, and underestimates the size of the test stimulus. This is supported by the fact that various cross-adaptations have been reported previously. For example, size adaptation reportedly affects perceptions of the number of objects (Zimmermann & Fink, 2016), whereas time and number cross-adaptation has also been noted (Tsouli, Dumoulin, Te Pas, & van der Smagt, 2019). Moreover, the direction in which stimuli are moving affects perceptions of the number of objects (Schwiedrzik, Bernstein, & Melloni, 2016). These cross-adaptations among time, numerosity, and size indicate a common mechanism underlying the estimation of magnitude. Recently, Petrizzo, Anobile, and Arrighi (2020) found that the adaptation to numerosity of action also distorted the perceived distance as the density-density adaptation does. Their finding suggests the visual distance is estimated in a higher stage dealing with the concept of numerosity. 

There may also be separable adaptation and perceptual size estimation stages. We assume that two stages are necessary to explain the density-size aftereffect. The first stage is density representation as a density metric in the visual sensory area. The second stage is the estimation processing based on that metric. The conceptual relationship between these stages is depicted in Figure 4. As mentioned in the Introduction, the metric could change due to a long exposure to a dense texture; in other words, adaptation occurs at the visual processing stage. The estimation stage is then related to the representation of common magnitude, indicating that magnitude estimation in one modality could affect that in other modalities. It should be noted here that adaptation occurs only during density processing. 

Figure 4.

 

Conceptual model of the functional relationship between the intraparietal sulcus (IPS) and visual sensory areas.

The relationship between density perception and numerosity perception has been the subject of debate for a long time. Barlow (1978) pointed out that density and numerosity perception are fundamentally different. Prior to Burr and Ross's study of numerosity adaptation, (Durgin & Huk, 1997; Durgin, 1995) examined the relationship between texture density adaptation (same as the density-density adaptation in the present study) and element numerosity perception. Durgin (1995) compared perceptions of density, number, and element distribution (cluster), especially after density-density adaptation. He found that density-density adaptation occurred in the range of 20 to 300 dots, while density adaptation decreased numerosity perception but did not affect cluster perception. In addition, Durgin and Huk (1997) showed that density-density adaptation was not affected by low-order image features such as spatial frequency and orientation. A series of studies by Durgin et al. suggested that density processing underlies numerosity perception. Dakin, Tibber, Greenwood, Kingdom, and Morgan (2011) investigated how numerosity and density estimations changed with a large number of elements (128). They showed that for density estimation, larger areas were associated with higher perceived density; similarly, in numerosity processing, larger areas were associated with larger number of perceived objects. Although this perceptual bias was lower for numerosity than density estimation, they argued that it is important that changes in stimulus size bring about a similar tendency. They proposed a model in which the output ratio of early multiple channel pairs of spatial frequency defines both density and numerosity estimates, which provides texture processing in early visual areas. These studies showed that density perception is not a concept that is completely contained in numerosity perception, and that density processing of elements may be faster than numerosity processing. 

Gebuis and Reynvoet (2013) analyzed the electroencephalograms (EEGs) of subjects when they were passively observing a texture composed of dots as used in numerosity adaptation studies and when they actively judged the number of dots. They found that neither active nor passive observation caused automatic numerosity processing. Activity of the parietal cortex may be important in counting task of the number of objects. A recent physiological study using EEG showed the contribution of the early visual cortex in detecting numerosity (Van Rinsveld, Guillaume, Kohler, Schiltz, Gevers, & Content, 2020). 

In response to these results, other studies assessed whether visual numerosity should be treated as a density-related phenomenon (Anobile, Cicchini et al., 2016; Pomè, Anobile, Cicchini, Scabia, & Burr, 2019). Three types of numbers were proposed as being important for estimation of the number of objects presented for a short duration: the subitizing range (1–4 or 1–5), the numerosity range, and the density range. The Weber ratios associated with numerosity and density perceptions were compared, and the cost of attention was analyzed (Anobile et al., 2014; Pomè et al., 2019). The results indicated that texture processing will be in operation when the number of dots exceeded 100 and the density exceeded 0.3 dots/deg². We used 144 dots for our higher-density stimulus (0.64 dots/deg²), which is sufficient for the texture processing described by Anobile, Cicchini, and Burr (2014). In addition, during presentation of the adapting stimulus, the subject was passively observing, that is, not attempting to count the adapting stimuli. During the adaptation period, “adaptation to density” (which is based on texture processing) occurred instead of “adaptation to magnitude.” Moreover, in another study, we showed that the density-size aftereffect arises from both retinotopic and spatiotopic representation effects (Hisakata & Kaneko, 2019). Processing occurring during magnitude estimation in the parietal cortex would reflect an adaptation effect on spatiotopic representations, similar to other cross-adaptations (Schwiedrzik et al., 2016). The above findings underlie our two-stage density-size aftereffect. 

A possible mechanism would be that a common system to code magnitude adapts to the temporal number of a particular presentation because it also estimates the temporal presentation number, and temporal duration could be coded there (Hayashi, Kanai, Tanabe, Yoshida, Carlson, Walsh, & Sadato, 2013; Hayashi, van der Zwaag, Bueti, & Kanai, 2018; Hayashi & Ivry, 2020). The temporal frequency and velocity of stimuli affect the perceived duration (e.g., Brown, 1995; Kaneko & Murakami, 2009; Linares & Gorea, 2015). For example, after a long exposure to flickering or moving stimuli with high temporal frequency, the magnitude estimation system is exhausted, and the output value would be smaller. If a common magnitude estimation system is used for spatial estimation, the perceived size would be affected by temporal refreshing of an adapting texture, resulting in a density-size aftereffect, as with the perceived duration. We observed this effect in the present results. We suppose that this indirect effect of temporal refreshing on spatial estimation is limited; therefore, the effect was saturated at a refresh rate of 2 Hz. On the other hand, the estimation of density itself is represented at the earliest stage of visual processing, in which spatial and temporal information are processed independently. Therefore, temporal refreshing did not modulate the aftereffect on density perception. 

In Experiment 2, refreshing at a relatively high frequency (15 Hz) induced a larger density-density aftereffect, whereas a low frequency did not. We speculate that the apparently high-density texture produced by refreshing with high frequency caused the greater density aftereffect. Because we used white and black dots on a gray background and the positions changed with a 15 Hz refresh rate, an afterimage might easily have remained (e.g., Hall & Wilsoncroft, 1964) and be treated as a member of the adapting dots, resulting in an increased density of the adapting texture. 

The refreshing of the adapting stimulus might increase the aftereffect by capturing the attention of the observer. Rezec, Krekelberg, and Dobkins (2004) investigated the effect of attention on the duration of the motion aftereffect and found that it was 1.4 times longer when attention was paid to the adapting stimulus than when it was not; this effect was not dependent on stimulus contrast. The density-size aftereffect was large for all stimuli above 2 Hz in Experiment 2 of this study, such that refreshing the stimulus may have captured the observer's attention. However, in Experiment 1, no refreshing effect was observed after 1 second adaptation (see Figures 2b,c), and the duration of the aftereffect was not extended by refreshing the adaptation stimulus after 1 or 5 seconds. The latency of exogenous attention, as reflected by saccades, is reportedly 230 to 250 ms (Şentürk, Greenberg, & Liu, 2016). If attention modulates the aftereffect, it should exert an effect even with a 1-second adaptation; however, the results did not show indicate this. In addition, if the adaptation effect is enhanced by increasing the salience of the visual stimulus, the refresh effect would likely be observed in the context of density–density adaptation; however, this was also not the case. Therefore, enhancement of the density–size aftereffect by refreshing the adaptation stimulus cannot be explained only by attention. 

In this study, we examined the relationship between density adaptation and the magnitude estimation system. We assumed that if the magnitude estimates of size are related to the density-size aftereffect, temporal modulation of an adaptation that can affect other modalities, such as numerosity adaptation, should also affect the density-size aftereffect. Here, the results showed that the perceived size shrinkage after adaptation to a dense texture increased as the frequency of temporal refreshing of an adapting texture increased, whereas this was not the case for the density-density aftereffect. From these results, we propose that basic density representation in the visual area acts as a density metric and the magnitude estimation system reads this density metric. For our next step, using functional magnetic resonance imaging, we will identify the brain area related to magnitude and density representations after adaptation. 

Acknowledgments

Supported by a Grant-in-Aid for Young Scientists (18K18341) and a Grant-in-Aid for Scientific Research (B) (20H01783). 

Author Contributions: R.H. was responsible for the conceptualization, methodology, software, investigation, visualization, writing of the original draft preparation, writing of and reviewing and editing, and funding acquisition. H.K. was responsible for the conceptualization, methodology, visualization, supervision, writing and reviewing, and editing. 

Commercial relationships: none. 

Corresponding author: Rumi Hisakata. 

Email: [email protected]. 

Address: School of Engineering, Tokyo Institute of Technology, 4-1-1 Kitakaname, Hiratsuka, Kanagawa 259-1292, Japan.