April 2019
Volume 19, Issue 4
Open Access
Article  |   April 2019
Visual orientation uncertainty in the rod-and-frame illusion
Author Affiliations
  • Antonella Pomante
    Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands
    a.pomante@donders.ru.nl
  • Luc P. J. Selen
    Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands
  • W. Pieter Medendorp
    Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands
Journal of Vision April 2019, Vol.19, 19. doi:10.1167/19.4.19
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Antonella Pomante, Luc P. J. Selen, W. Pieter Medendorp; Visual orientation uncertainty in the rod-and-frame illusion. Journal of Vision 2019;19(4):19. doi: 10.1167/19.4.19.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Estimation of the orientation of the head relative to the earth's vertical is thought to rely on the integration of vestibular and visual cues. The role of visual cues can be tested using a rod-and-frame task in which a global visual scene, typically a square frame, is displayed at different orientations together with a rod whose perceived direction is a proxy for the head-in-space estimate. While it is known that the frame biases this percept, and hence the subjective visual vertical, the possible role of the rod itself in this processing has not been examined. Current models about spatial orientation assume that the visual orientation of the rod and its uncertainty play no role in the visual-vestibular integration process, but are only involved in the transformation that yields rod orientation in space, thereby contributing additive noise to the subjective visual vertical. Here we tested the validity of this assumption in the rod-and-frame task by replacing the rod with an ellipse whose orientation uncertainty was manipulated by varying its eccentricity (i.e., making the ellipse more or less rounded). Using a psychophysical approach, subjects performed this ellipse-and-frame task for three different eccentricities of the ellipse (0.74, 0.82, 0.99) and three frame orientations (−17.5°, 0°, 17.5°). Results show that ellipse eccentricity affects the uncertainty but not the bias of the subjective visual vertical, suggesting that the ellipse does not interact with the frame in global visual processing but contributes additive noise in computing its orientation in world coordinates.

Introduction
Knowing how our head is oriented relative to the earth's vertical is necessary to maintain a correct posture, keep balance, and control body movements. The vestibular organs, the pressure sensors in the skin, the proprioceptors in the body and neck, and visual cues all contribute to building our sense of head orientation in space, or conversely the sense of what is upright. However, these signals are not all equally precise, and are expressed in different frames of reference. Therefore, to derive the best estimate of head orientation in space (Display Formula\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\({\tilde H_S}\)), they must be transformed into a common reference frame before they can be integrated in a precision-weighted manner (Clemens, De Vrijer, Selen, Van Gisbergen, & Medendorp, 2011; Ernst & Banks, 2002; Knill & Pouget, 2004; Kording & Wolpert, 2004; McGuire & Sabes, 2009). 
This computational approach relies on the concept of Bayesian inference, stating that perception depends on the statistical properties of the incoming signals together with prior assumptions based on earlier experience. Various studies have now used this framework to understand the brain's computations for spatial orientation and body-tilt perception (Alberts, De Brouwer, Selen, & Medendorp, 2016; De Vrijer, Medendorp, & Van Gisbergen, 2008, 2009; De Winkel, Katliar, Diers, & Bülthoff, 2018; Laurens & Droulez, 2007; MacNeilage, Banks, Berger, & Bülthoff, 2007; Tarnutzer, Bockisch, Straumann, & Olasagasti, 2009). 
Most of these studies have validated their model proposals experimentally by assessing spatial orientation using the Subjective Visual Vertical (SVV) task. In this task, participants are asked to judge the orientation of a visual line in space Display Formula\({\tilde L_S}\), which requires not only the Bayesian estimate of head orientation in space Display Formula\({\tilde H_S}\) but also internal estimates of eye-in-head orientation Display Formula\({\tilde E_H}\) and the orientation of the line on the retina Display Formula\({\tilde L_E}\), according to Display Formula\({\tilde L_S} = {\tilde H_S} + {\tilde E_H} + {\tilde L_E}.\) In this computation, it is generally assumed that the orientation of the visual line on the retina does not affect the multisensory computation of the head-in-space estimate, meaning that it only provides additive noise to the overall variance of the SVV. 
Generally, visual cues influence the percept of head orientation. For example, studies have shown that if the visual line is surrounded by a large visual frame, the SVV deviates toward the orientation of the frame (DiLorenzo & Rock, 1982; Ebenholtz, 1977; Vingerhoets, De Vrijer, Van Gisbergen, & Medendorp, 2008; Witkin & Asch, 1948; Zoccolotti, Antonucci, Goodenough, Pizzamiglio, & Spinelli, 1992). This phenomenon, known as the rod-and-frame effect, has been explained by an influence of the visual frame on the internal estimate of head orientation (Matin & Li, 1995), which in turn affects the setting of the rod. Importantly, a frame configuration appears not to be essential: Even a single peripheral line can bias the internal sense of head orientation (Li & Matin, 2005a, 2005b; Vingerhoets et al., 2008). If a peripheral line can act as a frame, we cannot simply rule out the possibility that the orientation of the visual line (i.e., rod) itself, to be judged in the rod-and-frame task, interacts with the other sensory signals that contribute to the internal estimate of head orientation, including other visual cues. 
Here we test whether the visual rod only introduces independent additive noise to the SVV (referred to as the addition hypothesis) or whether it induces a bias to the multisensory estimate of head orientation, and thus the SVV, through an interaction with the frame (here called the interaction hypothesis). Figure 1A provides a schematic representation of the putative computations: The focal visual line, as part of the visual scene, could interact with the global visual frame (gray arrow) and affect the visual contribution to head orientation in space Display Formula\({\tilde H_{{\rm{Svis}}}}\), as suggested by the interaction hypothesis, or only provide information about the retinal orientation of the line Display Formula\({\tilde L_E}\), as implied by the addition hypothesis. 
Figure 1
 
(A) Schematic representation of sensory integration for vertical perception. An optimal estimate of head orientation in space \({\tilde H_S}\) is obtained by integrating vestibular information and visual information. In a rod-and-frame task, it is normally assumed that visual cues about head orientation are provided only by a global visual frame, while the focal line gives only information about line-on-eye orientation \({\tilde L_E}\), which is added to eye-in-head \({\tilde E_H}\) and \({\tilde H_S}\) signals to estimate the required orientation of the line in space \({\tilde L_S}\). When the head is tilted, the vestibular-driven ocular counter-roll defines the orientation of the eye in head. According to the interaction hypothesis, the focal visual line interacts with the visual frame (as indicated by the gray arrow) and provides a direct visual contribution to head-in-space estimation \({\hat H_{Svis}}\). The resulting uncertainty is the result of optimal integration. According to the addition hypothesis, the focal visual line provides information only about retinal orientation of the line \({\tilde L_E}\); hence the variance of the visual ellipse is independent to the variance of the \({\tilde H_S}\) computation. (B) Qualitative model predictions. Left panels: Addition hypothesis. If the rod contributes to the estimation of the line in space in an additive manner, manipulation of the uncertainty of the focal visual line (here accomplished by varying the eccentricity of an ellipse) would affect the response uncertainty, with a larger effect when the frame is not tilted. No effect would be observed for the bias. Right panels: Interaction hypothesis. If the rod interacts with the frame, contributing to the visually derived head-in-space estimate, the uncertainty of the subjective visual vertical would also decrease with the eccentricity of the ellipse, but with a stronger effect for the tilted frame. Following the rules of multisensory integration, the different ellipse eccentricities would affect the modulation of the bias, whose amplitude would increase as the ellipse became more rounded (lower eccentricity). In the experiments, we probed three different eccentricities and three different frame orientations, so the hypotheses would show a difference in the slope of the lines connecting the points of interest (gray dashed lines).
Figure 1
 
(A) Schematic representation of sensory integration for vertical perception. An optimal estimate of head orientation in space \({\tilde H_S}\) is obtained by integrating vestibular information and visual information. In a rod-and-frame task, it is normally assumed that visual cues about head orientation are provided only by a global visual frame, while the focal line gives only information about line-on-eye orientation \({\tilde L_E}\), which is added to eye-in-head \({\tilde E_H}\) and \({\tilde H_S}\) signals to estimate the required orientation of the line in space \({\tilde L_S}\). When the head is tilted, the vestibular-driven ocular counter-roll defines the orientation of the eye in head. According to the interaction hypothesis, the focal visual line interacts with the visual frame (as indicated by the gray arrow) and provides a direct visual contribution to head-in-space estimation \({\hat H_{Svis}}\). The resulting uncertainty is the result of optimal integration. According to the addition hypothesis, the focal visual line provides information only about retinal orientation of the line \({\tilde L_E}\); hence the variance of the visual ellipse is independent to the variance of the \({\tilde H_S}\) computation. (B) Qualitative model predictions. Left panels: Addition hypothesis. If the rod contributes to the estimation of the line in space in an additive manner, manipulation of the uncertainty of the focal visual line (here accomplished by varying the eccentricity of an ellipse) would affect the response uncertainty, with a larger effect when the frame is not tilted. No effect would be observed for the bias. Right panels: Interaction hypothesis. If the rod interacts with the frame, contributing to the visually derived head-in-space estimate, the uncertainty of the subjective visual vertical would also decrease with the eccentricity of the ellipse, but with a stronger effect for the tilted frame. Following the rules of multisensory integration, the different ellipse eccentricities would affect the modulation of the bias, whose amplitude would increase as the ellipse became more rounded (lower eccentricity). In the experiments, we probed three different eccentricities and three different frame orientations, so the hypotheses would show a difference in the slope of the lines connecting the points of interest (gray dashed lines).
To distinguish between these two hypotheses, we used a rod-and-frame protocol in which we replaced the visual rod with a visual ellipse. The ratio of the main and minor axes of the ellipse defines its eccentricity (i.e., roundness) and hence the uncertainty by which subjects could estimate its orientation (i.e., if the ellipse is of pure circular shape, its orientation is ill defined). 
Figure 1B qualitatively illustrates the predictions for the two hypotheses in terms of bias and uncertainty of the SVV responses in this ellipse-and-frame task. For both hypotheses, the presentation of a more rounded ellipse (i.e., higher uncertainty) should result in an increase of the response uncertainty. However, if the noise associated with the retinal orientation of the ellipse is additive, the effect should be more pronounced when the frame is upright compared to the tilted frame (bottom left panel). Conversely, if the ellipse interacts with the frame in determining a percept of head orientation, the effect on the uncertainty should increase as the frame tilts away from 0° (bottom-right panel). As for the SVV bias, previous studies (Alberts et al., 2016) have shown a sinusoidal modulation with the frame orientation (top panels). While this modulation should not be affected if the ellipse information is additive (top left panel), an interaction between the rod and the frame should cause this modulation to change: When the orientation estimate of the ellipse becomes more uncertain, the biasing effect of the frame should increase, because the subject should rely more on the frame in the SVV judgment (top right panel). 
Results show that the manipulation of the eccentricity of the visual ellipse influences the uncertainty of the verticality judgments in a manner that mimics the hypothesis of additive noise, and does not modulate the SVV bias in the rod-and-frame task. This validates the basic assumption that the visual line does not interact with the frame and that its uncertainty contributes only additively to the estimation of the SVV. 
Methods
Subjects
Thirty-six individuals (21 female, 15 male; ages: 20–50 years) provided written informed consent to participate in the experiment. The study was approved by the ethics committee of the Faculty of Social Sciences of Radboud University, Nijmegen, the Netherlands. Subjects had normal or corrected-to-normal visual acuity and were free of any known vestibular disorders. Before the experiment began, they were carefully instructed about the task; no feedback about their performance was provided during the experiment. If we could not obtain reliable psychometric estimates in all conditions tested in a subject (see later for more details), all of that subject's data were excluded from further analyses. This left 25 complete data sets in the analyses. 
Experimental setup
Subjects were seated in a chair in front of an organic-LED TV screen (LG 55EA8809; 123 × 69 cm, 1,920 × 1,080 pixels, refresh rate: 60 Hz). A height-adjustable chin rest supported and fixed the head in a natural upright position. The center of the screen was positioned at eye level at a distance of 95 cm from the cyclopean eye. Stimuli were controlled using custom-written Python code. Except for the visual stimuli, the room was completely dark. Note that an organic-LED screen does not emit any light when a pixel is set to black. 
Experimental procedure
Subjects performed a rod-and-frame task in which we replaced the rod with an ellipse whose orientation uncertainty was manipulated by making its shape more or less rounded. While the major axis of the ellipse was maintained constant (subtending 12° visual angle), the minor axis was varied as to obtain ellipses with eccentricity values of 0.74 (near a circle), 0.82, and 0.99. The lower the eccentricity value, the more rounded the ellipse and thus the less precisely its orientation can be estimated. Only the circumference of the ellipse was drawn, in gray (1-mm line width) on the black background. The center of the ellipse remained black to keep the luminance low (<4 cd/m2) and avoid visual afterimages. A fixation dot (2-mm diameter, in gray) was displayed in the center of the screen and had to be fixated for the entire duration of the experiment. 
Each trial started by presenting a square frame—each edge subtending 18.3°, with a width of 0.22° (3 mm)—also drawn in gray. The frame was displayed in the center of the screen, in an orientation randomly chosen out of three possible angles (−17.5°, 0°, 17.5°; Figure 2A). The two tilted orientations are known to maximally bias the SVV (e.g., Alberts et al., 2016). After 250 ms, the ellipse was briefly flashed (one frame, i.e., 17 ms) in the center of the frame, with its major axis in an orientation determined by an adaptive psychometric approach (see later). Subjects had to indicate whether they perceived the orientation of the ellipse to be clockwise or counterclockwise with respect to the gravitational vertical, by pressing one of two buttons on a button box. After the response, the screen turned black for 500 ms, after which the next trial started (Figure 2B). 
Figure 2
 
Illustration of the experimental task. (A) Subject views a frame in an orientation of −17.5°, 0°, or 17.5°. The task is to judge whether a briefly flashed (17 ms) ellipse, drawn with different eccentricities (0.74, 0.82, 0.99), is oriented clockwise or counterclockwise with respect to the gravitational vertical. (B) Temporal sequence of a trial.
Figure 2
 
Illustration of the experimental task. (A) Subject views a frame in an orientation of −17.5°, 0°, or 17.5°. The task is to judge whether a briefly flashed (17 ms) ellipse, drawn with different eccentricities (0.74, 0.82, 0.99), is oriented clockwise or counterclockwise with respect to the gravitational vertical. (B) Temporal sequence of a trial.
The paradigm contained nine conditions, following from all combinations of the three frame orientations and the three eccentricity values of the visual ellipse. The nine conditions were randomly tested across trials, with a total of 150 trials per condition. For each condition, an independent, adaptive psychometric procedure determined the orientation of the major axis of the visual ellipse in the subsequent trial (see Adaptive stimulus section; see also Kontsevich & Tyler, 1999). The total experiment contained 1,350 trials and was run in ∼45 min. 
Adaptive stimulus selection
In order to efficiently—that is, with as few trials as possible—establish the bias and uncertainty of the SVV in each of the nine conditions, we used an adaptive algorithm (Ψ; Kontsevich & Tyler, 1999). This algorithm determines the optimal—that is, most informative—ellipse orientation for the upcoming trial for determining the parameters of the psychometric curve associated with a particular condition. More specifically, it selects the ellipse orientation that minimizes the expected entropy Display Formula\(E\left[ {{H_t}\left( x \right)} \right]{}\) (i.e., maximizes the information gain) over the parameters of interest. In our case, we assumed a cumulative Gaussian curve to describe the response data—that is, the likelihood of the observed response—so our parameters of interest are μ (bias) and σ (standard deviation). The expected information gain is computed by taking into account the probability of a clockwise or counterclockwise response to the stimulus based on the current beliefs about μ and σ. The likelihood and prior used to initialize the Ψ procedure were defined based on the following values for stimuli and parameters: Display Formula\(x \in \left[ { - 9^\circ ,9^\circ } \right]{}\) with 140 values, Display Formula\(\mu \in \left[ { - 7^\circ ,7^\circ } \right]{}\) with 100 values, and σ ∈ [e −5°, 5°] with 100 values. Thus, each condition was tested following its own independent adaptive procedure that ran in parallel with the conditions being tested. Unfortunately, this approach did not always converge on stable parameter values for each of the nine conditions. If the algorithm did not converge in one of the conditions, the subject's complete data set was removed from further analyses. 
Data analyses
Data analyses were conducted using MATLAB (MathWorks, Natick, MA). Although the Ψ algorithm provides an estimate of μ and σ on each trial based on the expected value of the current posterior probability, we decided to fit the parameters off-line: Since the posterior over μ and σ, as well as the possible stimulus values (i.e., the ellipse orientations), are discretized in the Ψ algorithm, an off-line fitting procedure, based on maximum likelihood, provides an estimate of μ and σ from a continuous domain. 
The bias and uncertainty of the SVV responses were estimated for each subject and each condition by fitting a cumulative Gaussian to the data points from the 150 trials. Using maximum likelihood, we estimated the parameters of the cumulative Gaussian (μ and σ) that maximize the likelihood of the observed stimulus-response pairs. We used repeated-measures analyses of variance (ANOVAs) in JASP (version 0.8.3.1; JASP Team) on the estimated bias and uncertainty values to evaluate our hypotheses. 
Results
Figure 3 shows the response data of a representative subject in each of the nine conditions, organized by frame orientation (columns) and ellipse eccentricity (rows). Each panel shows the proportion of clockwise responses (black circles) as a function of ellipse orientation relative to vertical. The size of the circles is directly proportional to the number of trials at which the corresponding stimulus was selected during the adaptive procedure. The solid lines in the various panels depict the cumulative Gaussian fit that matches the observed responses best. 
Figure 3
 
Psychometric curves obtained in an exemplar subject in each of the nine conditions tested. Curves were fitted based on 150 stimulus-response pairs; the size of the dots is proportional to the number of trials in which that particular ellipse orientation was probed. Notice how the bias (black vertical lines) shifts toward the direction of the frame, irrespective of ellipse eccentricity, whereas the slope of the psychometric curves increases with ellipse eccentricity.
Figure 3
 
Psychometric curves obtained in an exemplar subject in each of the nine conditions tested. Curves were fitted based on 150 stimulus-response pairs; the size of the dots is proportional to the number of trials in which that particular ellipse orientation was probed. Notice how the bias (black vertical lines) shifts toward the direction of the frame, irrespective of ellipse eccentricity, whereas the slope of the psychometric curves increases with ellipse eccentricity.
The point of subjective equality (the μ of the fitted cumulative Gaussian, i.e., the ellipse orientation at which subjects respond with clockwise in 50% of cases) denotes the bias in the SVV and is indicated by the horizontal and vertical dashed lines. As expected, the perceived vertical was biased toward the orientation of the frame (left and right columns) compared to the perceived vertical for the upright frame (center column). However, this bias did not show a systematic modulation with the eccentricity of the ellipse (rows), suggesting that the uncertainty of the ellipse orientation did not affect the SVV bias in this subject. The uncertainty of the SVV was inversely related to the slope of the psychometric curve. In this subject, the uncertainty in the SVV decreased (the psychometric curve became steeper) as the eccentricity of the ellipse increased (i.e., it became less rounded), especially when the frame was upright (middle column). 
Figure 4 illustrates the bias (left panel) and uncertainty (right panel) of the SVV for the individual subjects (gray symbols) as well as the average across subjects (M ± SE), as a function of frame orientation. Distinct symbols refer to the different ellipse eccentricities, and the dashed lines connect the SVV values for the same ellipse eccentricity. On average, when the visual frame was tilted −17.5° or 17.5° relative to the upright frame, the bias shifted in the direction of the frame by about 2°, as if the subject perceived that the head was tilted. There were no clear effects of ellipse eccentricity on this bias pattern. Indeed, a repeated-measures ANOVA with eccentricity and frame orientation as factors revealed a significant main effect of frame orientation on SVV bias, F(2, 48) = 45.62, p < 0.01, but no effect of ellipse eccentricity, F(2, 48) = 1.13, p = 0.33. We also used a repeated-measures ANOVA to examine the effects of frame orientation and ellipse eccentricity on SVV uncertainty in Figure 4 (right panel). This revealed a main effect of frame orientation, F(2, 48) = 28.15, p < 0.001, with an increase in SVV uncertainty for the tilted compared to the upright frame (paired t test, p < 0.001). The ANOVA also revealed that SVV uncertainty significantly increased with decreasing ellipse eccentricity, F(2, 48) = 3.66, p = 0.03, indicating that the manipulation was successful. The effect of the manipulation was only observed when the frame was upright, F(2, 48) = 16.51, p < 0.001; the uncertainty was not significantly affected with a tilted frame: at −17.5°: F(2, 48) = 0.45, p = 0.63; 17.5°: F(2, 48) = 2.82, p = 0.06. 
Figure 4
 
Bias (left panel) and uncertainty (right panel) of the subjective visual vertical for all subjects (N = 25, smaller gray markers) and their average (M ± SE) versus frame orientation. Symbols refer to the different ellipse eccentricities. Dashed lines connect the average values of subjective visual vertical for the same ellipse eccentricities.
Figure 4
 
Bias (left panel) and uncertainty (right panel) of the subjective visual vertical for all subjects (N = 25, smaller gray markers) and their average (M ± SE) versus frame orientation. Symbols refer to the different ellipse eccentricities. Dashed lines connect the average values of subjective visual vertical for the same ellipse eccentricities.
Thus, these results show that uncertainty about the orientation of the ellipse on the retina is only added to the uncertainty about head-in-space orientation, and that the ellipse does not interact with other visual cues in biasing the head-in-space orientation estimate itself. This supports the assumption that the bias in the SVV is not affected by the level of orientation uncertainty of the rod in the rod-and-frame task. 
Discussion
We tested whether the rod interacts with the frame in building an estimate of head orientation in space, and hence causing a bias in the subjective visual vertical. We replaced the classical rod with an ellipse whose eccentricity (roundness) we varied to manipulate the uncertainty in the orientation of its main axis. Results show that the effect of the ellipse eccentricity on the SVV uncertainty was greater when the frame was upright and that the bias in the SVV was not affected by this manipulation. This provides evidence that the ellipse does not interact with the frame as part of a global visual process that biases the estimation of the head-in-space orientation, and thus the SVV. This refutes our interaction hypothesis. Instead, our results favor the addition hypothesis by suggesting that the uncertainty about the ellipse orientation on the retina plays a role at a later processing stage, adding variance when ellipse orientation on the retina is combined with eye-in-head and head-in-space orientation signals to transform it into world coordinates. 
This assumption of additive retinal orientation noise was already incorporated in our models a decade ago (Alberts et al., 2016; Clemens et al., 2011; De Vrijer et al., 2008, 2009; Vingerhoets et al., 2008). In all our Bayesian optimal-integration models, an estimate of head-in-space orientation is constructed based on noisy information from the otoliths (De Vrijer et al., 2008, 2009), somatosensory organs (Alberts et al., 2016; Clemens et al., 2011), visual contextual information (Alberts et al., 2016; Vingerhoets et al., 2008), and prior experiences. In an SVV task, of which the rod-and-frame task is a special case, the head-in-space estimate has to be transformed into a probability distribution of how a vertical line will fall onto the retina, using both the eye-in-head orientation and the uncertainty about the line on retina. Because the line-on-retina uncertainty is small (Vandenbussche, Vogels, & Orban, 1986) compared to the uncertainty associated with other sensory modalities, it is normally omitted in the modeling. This choice is also made to reduce the number of parameters in the model and make the model tractable (see De Vrijer et al., 2009). However, this simplification also implies that the noise levels of the individual sensory signals are not distinguishable, at the expense of attributing the retinal uncertainty to other sensory systems involved in the integration process. Although we argue that these simplifying assumptions are justified when modeling the standard rod-and-frame task, such a Bayesian model, which assumes a noiseless signal about line orientation on the retina, is not warranted in modeling the current experiment: We artificially increased the line-orientation uncertainty, in an attempt to segregate the noise associated to the line on retina and study its effect on the percept of head orientation in the presence of contextual visual cues. 
In order to quantify the uncertainty of the ellipse-on-retina orientation, one would have to run a separate psychophysical experiment. For example, Vandenbussche et al. (1986) measured just-noticeable difference levels for orientation discrimination of <1°. In a preliminary experiment (see Supplementary File S1) leading up to the experiment presented here, we psychophysically tested how precisely subjects perceive the orientation of ellipses with different eccentricities (i.e., 0.6, 0.745, 0.788, 0.821, 0.846, 0.866, 0.968, 0.992, 0.997) in the absence of a visual frame. We observed a hyperbolically decreasing precision curve (see Supplementary Figure S1) that leveled off at an eccentricity of about 0.96 at a precision of about 0.7°, which is in the range reported by Vandenbussche et al. Based on the precision curve, we selected the three eccentricity values of 0.74, 0.82, and 0.99 for the main experiment, corresponding to 1.2°, 1.0°, and 0.7° uncertainty, respectively. As shown in Figure 4 (right panel), the decrease in perceptual uncertainty about the SVV with increasing eccentricity of the ellipse in the presence of an upright frame is in the same range. 
Our data are in line with the addition hypothesis: The effect of the eccentricity of the ellipse on the SVV uncertainty is larger when the frame is upright, and the effect is insignificant for the tilted frames. This could be due to the fact that the higher uncertainty induced by the frame completely masks the contribution of orientation uncertainty of the ellipse. As to the bias of the SVV, the modulation induced by the frame was on average 2° across all ellipse eccentricities. This limited effect is comparable to the previous findings of our group, using the same frame size and rod length (Alberts et al., 2016). Antonucci, Fanzon, Spinelli, and Zoccolotti (1995) have also reported such values, whereas Zoccolotti, Antonucci, Daini, and Martelli (1997) documented a slightly larger value of about 5°, perhaps reflecting different experimental factors (Zoccolotti et al., 1992). Importantly, the bias did not change with orientation uncertainty of the ellipse, indicating that visual orientation noise of the rod is more likely added to than integrated with a visual percept of head orientation in space. While we replaced the rod with a visual ellipse to probe spatial orientation, other studies have shown that orientations of objects, which are more important for perceptual recognition, are influenced more by visual context and body orientation (Dyde, Jenkin, & Harris, 2006). For future work, it would be interesting to manipulate the variability of such probes and see if they behave in the same way as the line probe investigated here. 
Acknowledgments
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No. 604063 (AP, WPM). The work has also been supported by the Netherlands Organization for Scientific Research, No. NWO-VICI 453-11-001 (WPM). 
Commercial relationships: none. 
Corresponding author: Antonella Pomante. 
Address: Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands. 
References
Alberts, B. B. G. T., de Brouwer, A. J., Selen, L. P. J., & Medendorp, W. P. (2016). A Bayesian account of visual-vestibular interactions in the rod-and-frame task. ENeuro, 3 (5): 0093–16.2016.
Antonucci, R., Fanzon, P. L., Spinelli, D., & Zoccolotti, P. (1995). Visual factors affecting the rod and frame illusion: Role of gap size and frame components. Perception, 24, 1105–1118.
Clemens, I. A. H., De Vrijer, M., Selen, L. P. J., Van Gisbergen, J. A. M., & Medendorp, W. P. (2011). Multisensory processing in spatial orientation: An inverse probabilistic approach. The Journal of Neuroscience, 31 (14), 5365–5377.
De Vrijer, M., Medendorp, W. P., & Van Gisbergen, J. A. M. (2008). Shared computational mechanism for tilt compensation accounts for biased verticality percepts in motion and pattern vision. Journal of Neurophysiology, 99 (2), 915–930.
De Vrijer, M., Medendorp, W. P., & Van Gisbergen, J. A. M. (2009). Accuracy-precision trade-off in visual orientation constancy. Journal of Vision, 9 (2): 9, 1–15, https://doi.org/10.1167/9.2.9. [PubMed] [Article]
De Winkel, K. N., Katliar, M., Diers, D., & Bülthoff, H. H. (2018). Causal inference in the perception of verticality. Science Reports, 8 (1), 1–12.
DiLorenzo, J. R., & Rock, I. (1982). The rod-and-frame effect as a function of the righting of the frame. Journal of Experimental Psychology, 8 (4), 536–546.
Dyde, R. T., Jenkin, M. R., & Harris L. R. (2006). The subjective visual vertical and the perceptual upright. Experimental Brain Research, 173, 612–622.
Ebenholtz, S. M. (1977). Determinants of the rod and frame effect: The role of retinal size. Perception and Psychophysics, 22 (6), 531–538.
Ernst, M. O., & Banks, M. S. (2002, January 24). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415 (6870), 429–433.
Knill, D. C., & Pouget, A. (2004). The Bayesian brain: The role of uncertainty in neural coding and computation. Trends in Neurosciences, 27 (12), 712–719.
Kontsevich, L. L., & Tyler, C. W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Research, 39 (16), 2729–2737.
Kording, K. P., & Wolpert, D. M. (2004, January 15). Baysian integration in sensorimotor learning. Nature, 427 (6971), 244–247.
Laurens, J., & Droulez, J. (2007). Bayesian processing of vestibular information. Biological Cybernetics, 96 (4), 389–404.
Li, W., & Matin, L. (2005a). The rod-and-frame effect: The whole is less than the sum of its parts. Perception, 34 (6), 699–716.
Li, W., & Matin, L. (2005b). Visually perceived vertical (VPV): Induced changes in orientation by 1-line and 2-line roll-tilted and pitched visual fields. Vision Research, 45 (15), 2037–2057.
MacNeilage, P. R., Banks, M. S., Berger, D. R., & Bülthoff, H. H. (2007). A Bayesian model of the disambiguation of gravitoinertial force by visual cues. Experimental Brain Research, 179 (2), 263–290.
Matin, L., & Li, W. (1995). Multimodal basis for egocentric spatial localization and orientation. Journal of Vestibular Research: Equilibrium and Orientation, 5, 499–518.
McGuire, L. M. M., & Sabes, P. N. (2009). Sensory transformations and the use of multiple reference frames for reach planning. Nature Neuroscience, 12 (8), 1056–1061.
Tarnutzer, A. A., Bockisch, C., Straumann, D., & Olasagasti, I. (2009). Gravity dependence of subjective visual vertical variability. Journal of Neurophysiology, 102 (3), 1657–1671.
Vandenbussche, E., Vogels, R., & Orban, G. A. (1986). Human orientation discrimination: Changes with eccentricity in normal and amblyopic vision. Investigative Ophthalmology & Visual Science, 27 (2), 237–245.
Vingerhoets, R., De Vrijer, M., Van Gisbergen, J. A. M., & Medendorp, W. P. (2008). Fusion of visual and vestibular tilt cues in the perception of visual vertical. Journal of Neurophysiology, 101 (3), 1321–1333.
Witkin, H. A., & Asch, S. E. (1948). Studies in space orientation: IV. Further experiments on perception of the upright with displaced visual fields. Journal of Experimental Psychology, 38, 762–782.
Zoccolotti, P., Antonucci, G., Goodenough, D. R., Pizzamiglio, L., & Spinelli, D. (1992). The role of frame size on vertical and horizontal observers in the rod-and-frame illusion. Acta Psychologica, 79 (2), 171–187.
Zoccolotti, P., Antonucci, G., Daini, R., & Martelli, M. L. (1997). Frame-of-reference and hierarchical-organisation effects in the rod-and-frame illusion. Perception, 26, 1485–1494.
Figure 1
 
(A) Schematic representation of sensory integration for vertical perception. An optimal estimate of head orientation in space \({\tilde H_S}\) is obtained by integrating vestibular information and visual information. In a rod-and-frame task, it is normally assumed that visual cues about head orientation are provided only by a global visual frame, while the focal line gives only information about line-on-eye orientation \({\tilde L_E}\), which is added to eye-in-head \({\tilde E_H}\) and \({\tilde H_S}\) signals to estimate the required orientation of the line in space \({\tilde L_S}\). When the head is tilted, the vestibular-driven ocular counter-roll defines the orientation of the eye in head. According to the interaction hypothesis, the focal visual line interacts with the visual frame (as indicated by the gray arrow) and provides a direct visual contribution to head-in-space estimation \({\hat H_{Svis}}\). The resulting uncertainty is the result of optimal integration. According to the addition hypothesis, the focal visual line provides information only about retinal orientation of the line \({\tilde L_E}\); hence the variance of the visual ellipse is independent to the variance of the \({\tilde H_S}\) computation. (B) Qualitative model predictions. Left panels: Addition hypothesis. If the rod contributes to the estimation of the line in space in an additive manner, manipulation of the uncertainty of the focal visual line (here accomplished by varying the eccentricity of an ellipse) would affect the response uncertainty, with a larger effect when the frame is not tilted. No effect would be observed for the bias. Right panels: Interaction hypothesis. If the rod interacts with the frame, contributing to the visually derived head-in-space estimate, the uncertainty of the subjective visual vertical would also decrease with the eccentricity of the ellipse, but with a stronger effect for the tilted frame. Following the rules of multisensory integration, the different ellipse eccentricities would affect the modulation of the bias, whose amplitude would increase as the ellipse became more rounded (lower eccentricity). In the experiments, we probed three different eccentricities and three different frame orientations, so the hypotheses would show a difference in the slope of the lines connecting the points of interest (gray dashed lines).
Figure 1
 
(A) Schematic representation of sensory integration for vertical perception. An optimal estimate of head orientation in space \({\tilde H_S}\) is obtained by integrating vestibular information and visual information. In a rod-and-frame task, it is normally assumed that visual cues about head orientation are provided only by a global visual frame, while the focal line gives only information about line-on-eye orientation \({\tilde L_E}\), which is added to eye-in-head \({\tilde E_H}\) and \({\tilde H_S}\) signals to estimate the required orientation of the line in space \({\tilde L_S}\). When the head is tilted, the vestibular-driven ocular counter-roll defines the orientation of the eye in head. According to the interaction hypothesis, the focal visual line interacts with the visual frame (as indicated by the gray arrow) and provides a direct visual contribution to head-in-space estimation \({\hat H_{Svis}}\). The resulting uncertainty is the result of optimal integration. According to the addition hypothesis, the focal visual line provides information only about retinal orientation of the line \({\tilde L_E}\); hence the variance of the visual ellipse is independent to the variance of the \({\tilde H_S}\) computation. (B) Qualitative model predictions. Left panels: Addition hypothesis. If the rod contributes to the estimation of the line in space in an additive manner, manipulation of the uncertainty of the focal visual line (here accomplished by varying the eccentricity of an ellipse) would affect the response uncertainty, with a larger effect when the frame is not tilted. No effect would be observed for the bias. Right panels: Interaction hypothesis. If the rod interacts with the frame, contributing to the visually derived head-in-space estimate, the uncertainty of the subjective visual vertical would also decrease with the eccentricity of the ellipse, but with a stronger effect for the tilted frame. Following the rules of multisensory integration, the different ellipse eccentricities would affect the modulation of the bias, whose amplitude would increase as the ellipse became more rounded (lower eccentricity). In the experiments, we probed three different eccentricities and three different frame orientations, so the hypotheses would show a difference in the slope of the lines connecting the points of interest (gray dashed lines).
Figure 2
 
Illustration of the experimental task. (A) Subject views a frame in an orientation of −17.5°, 0°, or 17.5°. The task is to judge whether a briefly flashed (17 ms) ellipse, drawn with different eccentricities (0.74, 0.82, 0.99), is oriented clockwise or counterclockwise with respect to the gravitational vertical. (B) Temporal sequence of a trial.
Figure 2
 
Illustration of the experimental task. (A) Subject views a frame in an orientation of −17.5°, 0°, or 17.5°. The task is to judge whether a briefly flashed (17 ms) ellipse, drawn with different eccentricities (0.74, 0.82, 0.99), is oriented clockwise or counterclockwise with respect to the gravitational vertical. (B) Temporal sequence of a trial.
Figure 3
 
Psychometric curves obtained in an exemplar subject in each of the nine conditions tested. Curves were fitted based on 150 stimulus-response pairs; the size of the dots is proportional to the number of trials in which that particular ellipse orientation was probed. Notice how the bias (black vertical lines) shifts toward the direction of the frame, irrespective of ellipse eccentricity, whereas the slope of the psychometric curves increases with ellipse eccentricity.
Figure 3
 
Psychometric curves obtained in an exemplar subject in each of the nine conditions tested. Curves were fitted based on 150 stimulus-response pairs; the size of the dots is proportional to the number of trials in which that particular ellipse orientation was probed. Notice how the bias (black vertical lines) shifts toward the direction of the frame, irrespective of ellipse eccentricity, whereas the slope of the psychometric curves increases with ellipse eccentricity.
Figure 4
 
Bias (left panel) and uncertainty (right panel) of the subjective visual vertical for all subjects (N = 25, smaller gray markers) and their average (M ± SE) versus frame orientation. Symbols refer to the different ellipse eccentricities. Dashed lines connect the average values of subjective visual vertical for the same ellipse eccentricities.
Figure 4
 
Bias (left panel) and uncertainty (right panel) of the subjective visual vertical for all subjects (N = 25, smaller gray markers) and their average (M ± SE) versus frame orientation. Symbols refer to the different ellipse eccentricities. Dashed lines connect the average values of subjective visual vertical for the same ellipse eccentricities.
Supplement 1
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×