Motion-form interaction: Motion and form aftereffects induced by distorted static natural scenes

Abstract

Spatially varying distortions (SVDs) are common artifacts of spectacles like progressive additional lenses (PALs). To habituate to distortions of PALs, the visual system has to adapt to distortion-induced image alterations, termed skew adaptation. But how this visual adjustment is achieved is largely unknown. This study examines the properties of visual adaptation to distortions of PALs in natural scenes. The visual adaptation in response to altered form and motion features of the natural stimuli were probed in two different psychophysical experiments. Observers were exposed to distortions in natural images, and form and motion aftereffects were tested subsequently in a constant stimuli procedure where subjects were asked to judge the skew, or the motion direction of an according test stimulus.

Exposure to skewed natural stimuli induced a shift in perceived undistorted form as well as motion direction, when viewing distorted dynamic natural scenes, and also after exposure to static distorted natural images. Therefore, skew adaptation occurred in form and motion for dynamic visual scenes as well as static images. Thus, specifically in the condition of static skewed images and the test feature of motion direction, cortical interactions between motion-form processing presumably contributed to the adaptation process.

In a nutshell, interfeature cortical interactions constituted the adaptation process to distortion of PALs. Thus, comprehensive investigation of adaptation to distortions of PALs would benefit from taking into account content richness of the stimuli to be used, like natural images.

Introduction

Progressive additional lenses (PALs) are commonly used optical elements for correction of presbyopia. They combine refractive corrections with an area of increased power in the lower area of the lens to support near vision in presbyopes. To avoid the disturbing optical inhomogeneity of bifocal spectacles, PALs have a continuous meridional power change (Sheedy, Campbell, King-Smith, & Hayes, 2005). As a consequence, a typical artifact of PALs are spatial geometrical distortions, the most prominent distortion of the lens being skew, a horizontally and vertically equal scale image shear (Habtegiorgis, Rifai, Lappe, & Wahl, 2017a; Meister & Fisher, 2008). The degree of distortion varies over the lens surface, showing an individual fingerprint depending on refractive error and lens design. In all PALs, distortion is most prominent in peripheral areas of such lenses (Barbero & Portilla, 2015). Optical distortions alter a variety of features of natural image content, such as position information, form, spatial frequency, luminance, contrast, orientation, texture, or optic flow signals (Bex & Makous, 2002; Bex, Mareschal, & Dakin, 2007; Bex, Solomon, & Dakin, 2009; Billock, De Guzman, & Scott Kelso, 2001; Dong & Atick, 1995; Heath, McCormack, & Vaughan, 1987). Through the dependency of optical distortions on position of the lens, in novel PAL wearers, objects appear distorted primarily when looking through peripheral parts of the lens (Gibson, 1937; Gibson & Radner, 1937). Upon head movement and gaze changes in the lens, artificial motion perception occurs (Kohler, 1962; Meister & Fisher, 2008; Welch, 1974; Welch, Carterette, & Friedman, 1978). However, after prolonged use, wearers report vanishing of side effects (Alvarez, Kim, & Granger-Donetti, 2017; Fonda, 1980; Kohler, 1962; Pick & Hay, 1964; Welch, 1974). Hence, habituation to PALs in the form of visual adaptation is hypothesized to contribute to restore undistorted vision. Also, other features, such as magnification, induce visual adaptation (Epstein, 1972; Kinney, Luria, & Weitzman, 1968; Ross, 1979; Vlaskamp, Filippini, & Banks, 2009). Visual adaptation is a process by which the visual system modifies its operating properties in response to changes in features (Clifford et al., 2007; Habtegiorgis, Rifai, Lappe, & Wahl, 2017b). How exactly the visual system adapts to distortion-induced alterations in the natural visual world is largely unknown. In PAL distortions specifically, due to the coexistence of prominent alterations of form and motion features, the contribution of adaptation in the respective features is unclear. 

Distortion-induced alteration (e.g., in form and motion features) possibly leads to plasticity in feature-selective neurons in the visual system. Livingstone and Hubel (1987) determined that there were two different and parallel working pathways in the visual system for form and motion information processing. However, neurophysiological and psychophysical studies provided evidence for the interaction between form and motion pathways (Barlow & Olshausen, 2004; Edwards & Crane, 2007; Fu, Shen, Gao, & Dan, 2004; Geisler, 1999; Geisler, Albrecht, Crane, & Stern, 2001; Grossberg, 1991; Mather, Pavan, Bellacosa, & Casco, 2012). Thus, understanding visual adaptation to PAL distortions in a natural environment requires the knowledge of distortion-induced plasticity in the motion and form visual pathways and their interaction. 

The present study uses short-term adaptation to skew distorted natural image content as a model system for visual adaptation as a habituation mechanism to PALs. Specifically, both form and motion adaptation were probed by appropriate test stimuli in response to dynamic as well as static distorted content. In the first experiment, form adaptation in natural images was compared in the presence and absence of motion information. Form adaptation still existed when motion information was removed. The second experiment evaluated adaptation of motion direction after exposure to distorted natural images. Motion adaptation was measured in the same conditions as in Experiment 1. Motion adaptation was induced by distorted natural image sequences and also after exposure to static distorted photographs. Hence, this study revealed unknown interfeature interaction between form and motion features during skew adaptation in natural scenes. 

Experiment 1: Form adaptation aftereffects induced by distorted form features of natural scenes

Habtegiorgis and colleagues (2017b) previously showed a form adaptation aftereffect (FAE) induced by distorted natural scenes. The aim of the present experiment was to examine whether form adaptation still exists when motion information is removed from such stimuli. Thus, form adaptation was measured in the presence and absence of motion information in two separate conditions: FAE after exposure to dynamic skewed natural image sequences (Condition 1) and FAE after exposure to static skewed natural images (Condition 2). In Condition 1, the FAE induced by both form and motion statistics of skewed natural image content represented a baseline to replicate previous results (Habtegiorgis et al., 2017b). In Condition 2, it was tested if FAE occurs in response to skewed natural image content and thus by form statistics only. 

Materials and methods

Study approval

The study was approved by the Ethics Committee of the Medical Faculty of the Eberhard Karls University of Tübingen and the University Hospital. 

Observers

Ten observers participated in this psychophysical experiment. Observers had normal or corrected-to-normal vision. All observers were naive about the objective of the experiment. In accordance with the Declaration of Helsinki, participants gave their informed written consent before participating in the experiment. 

Experimental setup

The study was designed in MATLAB (MathWorks, Natick, MA, USA) using Psychophysics Toolbox Version 3 (Brainard, 1997) on an apple computer (Apple, Cupertino, CA, USA). During the experiment, the stimuli were presented on a CRT monitor (HM204DT A; Iiyama) with a screen resolution of 1,280 × 1,024 pixels (406 × 304 mm). The screen refresh rate was 85 Hz. 

A chin- and headrest was used to stabilize the head of observers. The viewing distance was 60 cm to the screen. The whole screen subtended a visual angle (VA) of 37° horizontally and 28° vertically. The stimuli were presented at 0° eccentricity and subtended a visual angle of 20° both horizontally and vertically. The experiment was conducted in an entirely darkened room. Right or left keys of a keyboard were used to collect observers’ responses after adaptation. 

Stimuli

Images were skewed by a shear angle of θ by remapping pixel positions x and y to distorted positions x_d and y_d using geometrical skew transformation matrix M in Equations 1 and 2 (Habtegiorgis et al., 2017b).  

\begin{equation}\left( {\begin{array}{@{}*{1}{c}@{}} {{x_d}}\\ {{y_d}} \end{array}} \right) = M{\rm{*\;}}\left( {\begin{array}{@{}*{1}{c}@{}} x\\ y \end{array}} \right).\end{equation}

(1)

 

\begin{equation}M = {\rm{\;}}\left( {\begin{array}{@{}*{1}{c}@{}} {1{\rm{\;}}tan\theta }\\ {tan\theta \;{\rm{\;}}1{\rm{\;}}} \end{array}{\rm{\;}}} \right).\end{equation}

(2)

 

In each measurement condition, adapting stimuli were taken from two groups of skewed natural images, containing left-skewed images with a skew angle of θ =   − 25° and right-skewed images with a skew angle of θ =   + 25°. The image content was identical in the oppositely skewed adapting stimuli groups. Only the inner 650 × 650 pixels of each skewed image were used. Furthermore, a Hanning window weighting function of the second order was applied to the skewed image to blend out the sharp boundaries (Harris, 1978). The window was used as a weighting function and had the same size as the images (i.e., 650 × 650 pixels). The distorted and blended images subtended a visual angle of 20° × 20° in the horizontal and the vertical direction at zero eccentricity. 

In Condition 1, adapting stimuli contained temporally correlated skewed natural image sequences. In total, 27,000 sequential frames from three different open-source movies were skewed and applied as adapting stimuli (Baumann, 2010; Kluge, 2008; Roosendaal, 2012). These image sequences were rendered at 30 frames per second. Through the temporal correlation of frames, not only each image but also motion statistics were strongly influenced by the skew. A description of the resulting motion statistics can be found in Habtegiorgis, Jarvers, Rifai, Neumann, and Wahl (2019a). 

In Condition 2, adapting stimuli were static skewed natural images. A total of 130 photographs containing different natural scenes (e.g., buildings, animals, flowers, or people) were taken from the McGill-calibrated color image database and Olmos and Kingdom (2004). In this condition, image content differs frame by frame; thus, stimuli are temporally uncorrelated, and motion statistics is unaffected by the skew. 

Form adaptation aftereffects were tested with plaid checkerboards of the same size as the adapting stimuli (Habtegiorgis et al., 2017b). The plaid was constructed by superimposing identical contrast sinusoidal gratings orientated at –45° to the right and +45° to the left. The spatial wavelength of the component gratings corresponded to the dimension of the squares’ diagonals in the plaid. The unskewed diagonals of the checkerboard have a horizontal and vertical dimension of d₀ = 40 pixels subtending a VA of 1.24°. Different skew amplitudes were induced in the plaid by varying the wavelength of the component gratings. 

The dimensions of the plaids' opposite diagonals were defined by the wavelengths of the component gratings in the corresponding directions (i.e., d_right and d_left) and had the same value d₀ = 1.24° of VA for a square plaid. Skew was induced by adding or subtracting Δ from the wavelengths the component gratings, as in Equation 3. Nonzero Δ stretches the plaid diagonally and shears the zero-crossings of the squares in the plaid.  

\begin{equation}\begin{array}{@{}l@{}} {d_{right}} = \;{d_0} - \;\Delta \\ {d_{left}} = \;{d_0} + \;\Delta \end{array}\end{equation}

(3)

 

Thus, plaids are skewed to the left when Δ is positive and to the right when Δ is negative. The skew amplitude A_skew was quantified by the magnification in oblique directions, which was induced by either geometrically shearing or varying the diagonal dimensions, as presented in Equation 4.  

\begin{eqnarray}{A_{skew}}\left( {\Delta /\theta } \right) &=& 1 - \;\frac{{{d_{right}}}}{{{d_{left}}}} = 1 - \;\frac{{{d_0} - \Delta }}{{{d_0} + \;\Delta }} \nonumber\\ &=& 1 - \;\frac{{1 - \;\tan \theta }}{{1 + \tan \theta }}.\;\end{eqnarray}

(4)

 

Examples from the two groups of adapting stimuli together with test stimuli are shown in Figure 1. 

Figure 1.

 

(a) Example of a skewed natural image sequence used as adaptor in Condition 1 that is weighted by a Hanning window. Image skew is 25° and –25°, respectively. (b) Example of 25°/–25° skewed natural image used as adaptor in Condition 2 that is weighted by a Hanning window. Additional static images are shown in Supplementary Figure S1. (c) Example of plaid checkerboard test stimuli with left and right skew directions of –14.5% and 14.5%.

Procedure

Observers in this study were informed about the procedure and instructed on how to respond to a test stimulus with a keyboard before experiment start. Viewing was monocular with the right eye. Adaptation was tested alternately to oppositely skewed stimuli successively, first to left-skewed and then to right-skewed natural image sequences. Adaptation aftereffect was tested after each adaptation using the method of constant stimuli. Skew amplitudes ranging from –14.5% to 14.5% were used as test stimuli in 10 discrete equidistant steps with 10 repetitions of each test stimulus in random order. Overall, 100 responses were recorded per subject to compute the psychometric curve of each adapting skew direction. For each condition, the same psychophysical procedure was followed, except that the adapting stimuli differed in the two conditions. At first, adaptation was induced by a 3-s exposure to the adaptation stimulus. In Condition 1, the dynamic image sequence was applied, whereas a single image was shown in Condition 2. Next, one of the test stimuli was presented for 0.25 s. Then, observers responded whether the checkerboard was skewed to the left or right by pressing the left or right key on the keyboard, respectively. The sequence of adaptation exposure, test stimulus, and response phase was repeated, alternating for the two skew directions. Experimental sessions of Condition 1 and Condition 2 were separated by a minimum of 24 hr. 

Statistical data analysis

The percentage of rightward responses was computed at each test stimulus skew amplitude. A cumulative Gaussian function was used to fit the percentage of rightward responses as a function of skew amplitude of the test stimuli with asymptotes set free but assumed to be equal (Schütt, Harmeling, Macke, & Wichmann, 2016). The point of subjective equality (PSE; i.e., the skew amplitude that was perceived to be unskewed) was defined by the skew amplitude at 50% of rightward responses. The adaptation aftereffect was evaluated by the difference between the PSE of the left- and right-skew adaptation (ΔPSE). The ΔPSEs from all observers were averaged to estimate the overall adaptation aftereffect. The significance of perceptual shift was determined by applying a paired t test on the ΔPSEs. 

Results

Figure 2 shows psychometric functions of the average response of all observers measured in the two conditions. The percentage of rightward responses was computed as a function of test stimuli skew amplitude. Negative skew amplitudes corresponded to left-skewed plaid checkerboards, and positive skew amplitudes corresponded to right-skewed plaid checkerboards. The graphs show data together with confidence intervals at PSE in blue for the right-skew adaptation aftereffect and in green for the left-skew adaptation aftereffect. 

Figure 2.

 

FAE from all the observers induced by dynamic skewed natural image sequences in Condition 1 (a) and by static skewed natural images in Condition 2 (b). The first column shows data, sigmoid fits, and confidence intervals at PSE in green for the left-skew adaptation aftereffects and in blue for the right-skew adaptation aftereffects. Negative stimulus skew amplitudes refer to left-skewed stimuli, and positive skew amplitudes refer to right-skewed stimuli. The bar plots in the right column show the average ΔPSE of all observers. The error bars show the standard error.

In both conditions, the PSE shifted toward the direction of skew after left- and right-skew adaptation. After exposure to left-skewed stimuli, observers perceived a left-skewed plaid checkerboard as unskewed and vice versa. In the left column of Figure 2, the average ΔPSEs over all the observers are shown. ΔPSE of Condition 1 was 7.18% ± 1.69%; ΔPSE of Condition 2 was 7.87% ± 1.37%. The effect sizes of the ΔPSEs measured in Conditions 1 and 2 were significantly different from zero, p < 0.05. 

In sum, adaptation aftereffects were observed within both conditions. Moreover, stimuli with only form content induced a similar FAE as stimuli containing form and motion features. 

Experiment 2: Motion adaptation aftereffects induced by distorted form features of natural scenes

Materials and methods

The materials and methods used in this experiment were the same as in Experiment 1, except for modifications described below. Motion content of the adaptation stimulus has been thoroughly described in Habtegiorgis et al. (2019a). 

Study approval

The study was approved by the Ethics Committee of the Medical Faculty of the Eberhard Karls University of Tübingen and the University Hospital. 

Observers

The same participants as in Experiment 1 participated in this psychophysical experiment. Participants gave their informed written consent in accordance with the Declaration of Helsinki prior to the experiment. 

Test stimuli

In both measurement conditions, test stimuli were dynamic random dots. Each dot was circular and subtended a VA of 0.14°. The annulus in which the dots were moving subtended a VA of 15° × 15° and always consists of 2,000 dots. In the center of the annulus, a red point was presented for central fixation. The dots moved coherently at a speed of 4°/s. The direction of the coherent motion was diagonally up or down at an angle of θ_tilt from the horizontal. For a specific motion direction, θ_tilt, the position of the dots x₁ and y₁ was updated to x₂ and y₂ in the subsequent frame using Equation 5. Positive θ_tilt corresponded to upward motion and negative to downward motion. The test stimulus to measure motion aftereffects is presented in Figure 3.  

\begin{equation}\frac{{{x_2}}}{{{y_2}}} = \;\frac{{\cos \theta \cdot {x_1}}}{{\sin \theta \cdot {y_2}}}.\;\end{equation}

(5)

 

Figure 3.

 

Dynamic random dot test stimuli for measurement of motion direction aftereffects. Illustration of (a) upward and (b) downward motion direction.

Procedure

In this experiment, the test stimulus was presented with 12 different motion directions, ranging from –6.6° to 6.6° in steps of 1.2°. Each test stimulus was repeated 10 times. Thus, 120 responses were recorded to fit the psychometric function of each adapting skew direction. Observers responded whether the dots were moving up or down by pressing the up or down keys on the keyboard, respectively. The order of the test stimuli of different motion directions was randomized. Experimental sessions of Condition 1 with skewed image sequences and Condition 2 with skewed static images as adaptors were separated by a minimum of 24 hr. 

Statistical data analysis

The percentage of upward responses was fitted as a function of test motion direction with the asymptotes set free but assumed to be equal (Schütt et al., 2016). The PSE (i.e., the motion direction that was perceived as horizontal) was defined by the motion direction at 50% of upward responses. The adaptation aftereffect was evaluated by the difference between the PSE of the left- and right-skew adaptation (ΔPSE). The ΔPSEs from all observers were averaged to estimate the overall adaptation aftereffect. The significance of perceptual shift was determined by applying a paired t test on the ΔPSEs. 

Results

Figure 4 shows the psychometric fits of the average response over all observers, separately for the two conditions. The percentage of upward responses is shown as a function of test stimulus direction. Negative motion directions correspond to downward motion, and positive motion directions correspond to upward motion. 

Figure 4.

 

MAE from all observers induced by dynamic skewed natural image sequences in Condition 1 (a) and by static skewed natural images in Condition 2 (b). The first column shows psychometric curves of the average response of all the observers. Sigmoid fit, data, and confidence intervals at PSE are shown in green for the left-skew adaptation aftereffects and in blue for the right-skew adaptation aftereffects. Positive θ_tilt corresponded to upward motion and negative to downward motion. In the second column, bar plots show the average ΔPSE of all the observers. The error bars show the standard error.

In both Condition 1 containing image sequences and Condition 2 containing static images as adaptors, the PSE shifted toward the adapting skew direction. Observers perceived an upward motion as a horizontal movement after adaptation by right-skewed images and vice versa. ΔPSE of Condition 1 was 2.78° ± 0.77°, and ΔPSE of Condition 2 was 4.49° ± 1.46°. The ΔPSEs from all observers were significantly different from zero in both conditions, t test <0.05. The corresponding averages are shown in the second column of Figure 4. 

Discussion

This study demonstrated motion direction adaptation as well as form adaptation in response to dynamic as well as static distorted natural images. Two different psychophysical experiments were conducted, and adaptation aftereffects were estimated by the induced shift in perception (PSE) in the two features of form and motion. Each feature was assessed by a feature-typical test stimulus. In the first experiment, distortion-induced form adaptation was tested on skewed checkerboards, and in the second experiment, distortion-induced motion direction adaptation was tested on dynamic random dot patterns. Condition 1 confirmed the current literature on adaptation to skewed dynamic natural stimuli (Habtegiorgis, Jarvers, Rifai, Neumann, & Wahl, 2019b; Habtegiorgis, Rifai, Lappe, & Wahl, 2017c). But as a novelty, in Condition 2, both form adaptation and motion direction adaptation were induced in the absence of motion information. These results revealed that motion direction aftereffect and form aftereffect are induced even upon exposure to static distorted natural scenes. Particularly surprising, in Experiment 2, Condition 2, motion direction adaptation was shown in response to static distorted images. Thereby, an interaction of form and motion pathways in the adaptation process was revealed. 

Interaction between form and motion direction processing neurons exists on different levels in the hierarchy (Mather, Pavan, Bellacosa Marotti, Campana, & Casco, 2013). Direction-selective neurons extract motion orthogonal to preferred orientation already in V1 (De Valois, William Yund, & Hepler, 1982; Hubel & Wiesel, 1968). But evidence from psychophysics suggests that motion-form interactions also take place at a level of optic flow integration (Pavan, Marotti, & Mather, 2013). Even heading estimation has been shown to be impacted not only by motion but also by form features (Niehorster, Cheng, & Li, 2010). Thus, adaptation to static skew is likely to contribute to the changes in estimating motion direction (Fu et al., 2004; Mather et al., 2012). As natural image content contains lower- and higher-level features, one might even speculate on an interaction at the level of object perception (McGraw, Whitaker, Skillen, & Chung, 2002; Nishida & Johnston, 1999; Snowden, 1998) and that the interaction might be located in area MT where motion information is processed (McGraw, Walsh, & Barrett, 2004). 

Presbyopes wearing PALs for the first time often complain of distortion and image swim until they familiarize to PALs (Ahmad Najmee, Buari, Mujari, & Rahman, 2017; Pope, 2000; Shagufta & Rabia, 2017; Sullivan, 1990; Sullivan & Fowler, 1989). Thus, distortions show the potential to challenge daily life with PALs (e.g., during climbing stairs; Johnson, Buckley, Scally, & Elliott, 2007). 

The experimental paradigm demonstrated visual adaptation after exposure to image skew on a time scale of minutes and the specific role of the form feature. In a similar paradigm, adaptation aftereffects to skewed image content have been demonstrated to show long-term effects in the range of months after exposure of minutes, thus well connecting to the broad time range reported on PAL habituation (Ahmad Najmee et al., 2017; Habtegiorgis, Rifai, Lappe, & Wahl, 2018; Pope, 2000; Shagufta & Rabia, 2017). 

While the experimental paradigm focuses on foveal aftereffects, transsaccadic transfer of aftereffects demonstrated the nonfoveal occurrence of aftereffects in a comparable paradigm, as they might potentially contribute to PAL habituation (Habtegiorgis, Rifai, & Wahl, 2018). 

Here, adaptation of the features of form and motion in response to image skew was demonstrated together with an interaction of the two features. Specifically, a dominant role of form information was revealed, inducing form adaptation as well as motion direction adaptation. In sum, motion and form pathways contribute to the adaptation process to distortions and presumably interact during adaptation. For further studies on habituation in PALs, distorted images provide a useful abstraction level as adapting stimuli in experimental paradigms. Thus, the present study provides evidence that exposure to static image skew serves as an experimental paradigm for the induction and analysis of perceptual habituation to PALs. 

Acknowledgments

Commercial relationships: KW and SW are employees of Carl Zeiss Vision International GmbH. 

Corresponding author: Katharina Rifai. 

Email: [e-mail]. 

Address: Institute for Ophthalmic Research, University of Tübingen.