Can walking motions improve visually induced rotational self-motion illusions in virtual reality?

Abstract

Illusions of self-motion (vection) can provide compelling sensations of moving through virtual environments without the need for complex motion simulators or large tracked physical walking spaces. Here we explore the interaction between biomechanical cues (stepping along a rotating circular treadmill) and visual cues (viewing simulated self-rotation) for providing stationary users a compelling sensation of rotational self-motion (circular vection). When tested individually, biomechanical and visual cues were similarly effective in eliciting self-motion illusions. However, in combination they yielded significantly more intense self-motion illusions. These findings provide the first compelling evidence that walking motions can be used to significantly enhance visually induced rotational self-motion perception in virtual environments (and vice versa) without having to provide for physical self-motion or motion platforms. This is noteworthy, as linear treadmills have been found to actually impair visually induced translational self-motion perception (Ash, Palmisano, Apthorp, & Allison, 2013). Given the predominant focus on linear walking interfaces for virtual-reality locomotion, our findings suggest that investigating circular and curvilinear walking interfaces offers a promising direction for future research and development and can help to enhance self-motion illusions, presence and immersion in virtual-reality systems.

Introduction

Facilitating natural and unrestrained exploration of large virtual environments remains one of the fundamental challenges in virtual reality (VR) research. The primary difficulty lies in supporting exploration of virtual environments that are much larger than the available physical space, preferably at costs which make it a practical solution for a wide audience. 

Physical walking in tracked spaces enables a high degree of kinematic similarity to real-world walking (Whitton et al., 2005), improves the sense of presence in virtual environments (Usoh et al., 1999), and facilitates navigation and acquisition of spatial information (Ruddle & Lessels, 2009; Ruddle, Volkova, & Bülthoff, 2011; Suma et al., 2010; Zanbaka, Lok, Babu, Ulinski, & Hodges, 2005). However, physical walking in large virtual environments requires large, unobstructed physical spaces equipped with a tracking system. Redirected walking techniques attempt to overcome this limitation by using inconspicuous perceptual manipulations to overtly or covertly steer users so they remain within the confines of the available space (Nescher, Huang, & Kunz, 2014; Razzaque, Kohn, & Whitton, 2001; Steinicke et al., 2009; Zmuda, Wonser, Bachmann, & Hodgson, 2013). It has been demonstrated, for example, that a 35-m × 35-m area is sufficient for redirecting users on a circular trajectory, while they believe they are treading upon virtually endless straight paths (Hodgson & Bachmann, 2013). An alternative approach is to use motion platforms or omnidirectional treadmills (Steinicke, Vissell, Campos, & Lecuyer, 2013), though such techniques are technically complex and can be prohibitively expensive. Both of these approaches require reliable prediction of the user's future path, which, in the general case, remains an open issue. Consequently, natural and unencumbered walking in VR remains a challenge (Steinicke et al., 2013). 

When cost and space availability constrain the use of physical walking techniques, it may be advantageous to focus instead on sensory stimulation to induce a compelling, embodied illusion of self-motion, or vection, without physically moving the observer (Hettinger, 2002; Riecke, 2011; Riecke & Schulte-Pelkum, 2013). Researchers have long been interested in such embodied self-motion illusions (Andersen, 1986; Dichgans & Brandt, 1978; Hettinger, 2002; Riecke, 2011; Väljamäe, 2009; Warren & Wertheim, 1990), which can be compelling to a point where one can no longer easily distinguish actual self-motion from illusory self-motion (Berthoz, Pavard, & Young, 1975; Brandt, Dichgans, & Koenig, 1973; Dichgans & Brandt, 1978; Held, Dichgans, & Bauer, 1975). Correlation between measures of vection and presence in virtual environments (Riecke & Schulte-Pelkum, 2014; Riecke, Schulte-Pelkum, Avraamides, Heyde, & Bülthoff, 2006) suggests that self-motion illusions can both enhance existing locomotion techniques and potentially lead to the development of compelling and inexpensive locomotion interfaces that do not require physical rotation or translation of the user (Riecke & Schulte-Pelkum, 2013; Riecke, Schulte-Pelkum, Caniard, & Bülthoff, 2005). 

Just as in the real world, a complete VR locomotion system consists of both translational and rotational components. In this study, however, we were interested in exploring the means of inducing circular vection, i.e., the illusion of self-rotation. Behavioral studies suggest that it is much harder to imagine and spatially update rotational self-motion in comparison to translational self-motion (Klatzky, Loomis, Beall, Chance, & Golledge, 1998; Presson & Montello, 1994; Rieser, 1989). The same is true in VR, as users experiencing only visually simulated self-motion tend to have difficulties updating rotations but not translations (Klatzky et al., 1998). As a result, the absence of physical rotation can negatively impact the ability to maintain orientation in space (Farrell & Robertson, 1998; Klatzky et al., 1998). Circular vection may help solve this problem, because it has been shown to facilitate perspective switches and orientation in the absence of physical rotation (Riecke, Feuereissen, Rieser, & McNamara, 2012). This suggests that the mere illusion of rotational self-motion may, at least to a certain degree, be able to compensate for the lack of physical motion. 

In VR simulations, the quality of self-motion illusions may depend upon sensory stimulation across a combination of multiple sensory modalities. Previous research shows that multisensory integration plays an important role in human motion perception, and that these cross-modal effects can be very complex (Soto-Faraco, Kingstone, & Spence, 2003). However, while self-motion illusions are generally enhanced if multiple feedback modalities are combined (Hettinger, Schmidt, Jones, & Keshavarz, 2014; Riecke, 2011; Riecke & Schulte-Pelkum, 2013), this is not always the case. For example, recent studies have reported that a combination of viewing forward motion visuals with matching biomechanical cues from walking on a linear treadmill resulted in significantly reduced sensations of forward self-motion when compared with viewing visual forward motions while standing still (Ash, Palmisano, Apthorp, & Allison, 2013; Kitazaki, Onimaru, & Sato, 2010; Onimaru, Sato, & Kitazaki, 2010). When applied correctly, multisensory stimulation can contribute to a higher sense of presence in VR (Dinh, Walker, Hodges, Song, & Kobayashi, 1999) and facilitate certain types of skill acquisition in virtual training scenarios (Ström et al., 2006). So understanding which multimodal combinations have a positive influence on vection is important from both the theoretical perspective of multisensory cue integration and an applied perspective of optimizing self-motion simulations in VR. 

Our research explores how combining visual stimulation with other senses may enhance the experience of vection by reducing the onset latency and increasing the intensity and convincingness of the users' experience of self-motion. Here we investigate whether a combination of visual and walking cues may contribute to providing VR users with a compelling sensation of self-rotation, thus reducing the need for actual self-motion. 

Theoretical background and motivation

Self-motion illusions have long fascinated humans (Helmholtz, 1866; Mach, 1875; Wood, 1895), likely because they are one of the few truly embodied and visceral illusions induced by visual stimuli. It is possible to induce such illusions using one or a combination of several kinds of sensory stimulation, including visual and biomechanical modalities. In the following sections, we first describe vection illusions induced by visual or biomechanical feedback alone and then discuss the effects of multimodal contributions to vection illusions. 

Visually induced vection

Visually induced self-motion illusions are probably the best-known and most researched kind of vection. They are based on the finding that when a moving visual stimulus covers a large portion of visual field, stationary observers can experience a compelling sensation of self-motion (Brandt et al., 1973; Dichgans & Brandt, 1978). This is exemplified by the “train illusion,” which many readers have likely experienced themselves. While seated on a stationary train waiting to depart and observing a train departing from the neighboring track, we often (erroneously) perceive that our own train has begun to move instead of the adjacent train (Helmholtz, 1866). This phenomenon is termed linear vection, though similarly compelling self-motion illusions can be induced for rotations (circular vection). For example, an observer seated inside a large textured cylinder (known as an optokinetic drum) rotating about the earth-vertical axis reliably experiences a strong sensation of self-rotation (Brandt et al., 1973; Dichgans & Brandt, 1978). More recently, computer-generated graphics and VR displays have also been shown to elicit vection, especially if the moving visual stimulus covers a sufficiently large field of view (FOV; for reviews, see Hettinger, 2002; Palmisano, Allison, Kim, & Bonato, 2011; Riecke, 2011; Riecke & Schulte-Pelkum, 2013). Until recently, however, most vection researchers have informally observed that reliably inducing vection illusions in head-mounted displays (HMDs) was not possible, as at the time most HMDs provided relatively narrow FOVs. For this study we used a relatively wide-FOV HMD (102° × 64°), which enabled us to demonstrate that circular vection can indeed be reliably induced with HMDs given a sufficiently large FOV. 

While it is problematic to directly compare visually induced linear vection with circular vection, as it is debatable how one might equate the amount of optic flow, in general, circular vection around the earth-vertical axis tends to be more convincing and intense and occurs earlier than linear forward or backward vection (Trutoiu, Mohler, Schulte-Pelkum, & Bülthoff, 2009). Furthermore, curvilinear trajectories seem to be similarly effective in inducing vection as simple rotations, which has implications for many VR applications and might be used to enhance translational motions. Whereas linear forward and backward vection is typically not as compelling as circular or curvilinear vection (Trutoiu et al., 2009), vertical up and down or “elevator” vection can be quite compelling, likely because of the reduced visual-vestibular cue conflict if the direction of simulator self-motion is aligned with the direction of gravity (Giannopulu & Lepecq, 1998; Kano, 1991; Trutoiu et al., 2009). 

It should be noted that the illusion of self-motion builds up gradually and lags behind the onset of visual motion. This onset latency can range from a few seconds to more than 20 s (Brandt et al., 1973; Dichgans & Brandt, 1978). The time course for other types of vection, such as auditory and walking-induced (biomechanical) vection, follows a similar pattern of delayed onset and gradual increase. 

Walking-induced (biomechanical) vection

Circular vection can also be induced without any additional stimulation by asking stationary blindfolded observers to step along a rotating floor plate similar to a small carousel (Becker, Raab, & Jürgens, 2002; Bles, 1981; Bles & Kapteyn, 1977; Bruggeman, Piuneu, Rieser, & Pick, 2009). Figure 1 illustrates an example of such a “circular treadmill,” where participants are asked to step along sideways with the rotating floor beneath their stationary seat. This often elicits walking-induced vection, also known as “biomechanical” (Bruggeman et al., 2009; Riecke, Feuereissen, Rieser, & McNamara, 2011), “apparent stepping around” (Bles, 1981; Bles & Kapteyn, 1977), or “podokinetic vection” (Becker, Nasios, Raab, & Jürgens, 2002). Biomechanical circular vection tends to be compelling to a point where the illusion sometimes cannot be distinguished from actual stepping around (Bles & Kapteyn, 1977). For example, DiZio and Lackner state, “Even though subjects know the platform can rotate under their feet, they still come to perceive it to be stationary and themselves as turning” (2001, p. 127). 

Figure 1

 

Experimental setup, showing the seated stationary participant stepping along the rotating circular treadmill. The head-mounted display (HMD) depicts the visual stimulus in the visual-only and bimodal conditions. For the biomechanical-only condition, the HMD was switched to black and participants were instructed to close their eyes. See included video or http://youtu.be/_vIVqpWINF0 for an illustration of the three stimulus combinations. A handheld joystick was used to indicate the onset, intensity, and convincingness of the vection percept.

Multimodal contributions to vection

Multisensory stimulation may decrease vection onset time and significantly enhance the illusion of self-motion. For example, a recent study shows that biomechanical circular vection can be significantly enhanced by providing concomitantly rotating sound fields presented via headphones (Riecke et al., 2011). In fact, there seems to be a larger benefit for adding spatialized sound to enhance biomechanical vection as compared to visually induced vection (Riecke et al., 2011; Riecke, Väljamäe, & Schulte-Pelkum, 2009). 

Although both visual and biomechanical cues are known to provide compelling circular vection by themselves, there is little research investigating their joint contribution to vection in more depth. DiZio and Lackner (2002) have remarked that participants in an optokinetic drum surrounding a circular treadmill can experience compelling and immediate circular vection if the floor and optokinetic drum rotate in sync. Under such conditions of full-field visual stimulation and walking in circles, participants can apparently experience saturated vection in the sense that they (erroneously) perceive the floor and walls as stationary and themselves as walking on a stationary platform inside a stationary optokinetic drum (DiZio & Lackner, 2002; Lackner & DiZio, 1988). Perceived turning velocity has been observed to be slightly but insignificantly overestimated (33°/s compared to the actual velocity of 24°/s; Lackner & DiZio, 1988). However, neither onset latency, intensity, nor convincingness of vection was assessed in any of these studies. 

In fact, there are surprisingly few studies that have explicitly assessed circular vection induced by the combination of visual and biomechanical cues. More than 30 years ago, Bles (1981) investigated how biomechanical cues (from stepping along off-center on a circular treadmill) interacted with visual cues (provided by a rotating optokinetic drum) and vestibular cues (from actual off-center rotations). All participants experienced circular vection in the visual, biomechanical, and bimodal conditions. Participants were asked to report whenever they had rotated through 360°, which was used to determine their average perceived turning velocity. This perceived self-rotation velocity was overall veridical and did not depend on whether participants received vestibular, biomechanical, or visual stimulation or any combination thereof. Although Bles states that the findings “suggest a possible delay in the onset of the subjective rotation in the apparent stepping around condition” (p. 53), unfortunately the study did not explicitly assess the onset latency, intensity, or convincingness of vection for any of the stimulus conditions examined. 

Deepening our understanding of how visual and walking cues in combination contribute to providing a compelling sensation of self-motion would not only be of theoretical interest and help to close a gap in the literature; it could also be relevant from the applied perspective of designing affordable yet effective means of enhancing self-motion perception, and thus potentially improve presence and immersion and the overall believability and effectiveness of a VR simulation (Hettinger, 2002; Riecke et al., 2005; Riecke & Schulte-Pelkum, 2013, 2014). 

To this end, we designed a study to investigate circular vection induced by (a) naturalistic visual cues presented via head-mounted display (HMD), (b) biomechanical cues from stepping along a rotating floor plate, and (c) the combination of visual and biomechanical cues. To provide a more comprehensive assessment, we combined multiple vection measures (onset latency, intensity, and convincingness) with a postexperimental debriefing and additional participant data, such as experience with 3-D computer games. Based on the literature we have discussed, we hypothesized that visual and biomechanical cues should each be able to reliably induce compelling vection in most if not all observers. Furthermore, based on informal observations by Lackner and DiZio (DiZio & Lackner, 2002; Lackner & DiZio, 1988), we hypothesized that combining visual and biomechanical cues would enhance vection, though there appears to be no prior reference for the amount of facilitation to be expected. 

Methods

Experimental design

Using a within-subject design, the experiment presented participants with three rotation conditions capable of eliciting illusory self-rotation. These conditions consisted of either only rotating visual stimuli (visual-only), only rotating biomechanical stimuli (biomechanical-only), or both visual and biomechanical stimuli rotating in synchrony (bimodal). Participants completed four repetitions for each of these stimulus conditions, resulting in three blocks of four trials, for 12 trials in total. Before the main experiment, participants also completed four practice trials. 

Each trial lasted 45 s, with a short break in between each pair of four-trial blocks to prevent motion sickness. The ordering of the three stimulus conditions was counterbalanced between participants, and each trial alternated between left and right rotation in an effort to reduce motion aftereffects and motion sickness. In each ordering of conditions, the group of participants consisted of two men and one woman. Two participants in each group were undergraduate students and one was a graduate student. 

Participants

A total of 18 participants (six women) completed the experiment. Participants were either undergraduate (12) or graduate (six) students at a Canadian university. Undergraduate students were recruited via an online research pool and were compensated with research credit for use in their coursework. Graduate students were recruited via word of mouth and were offered $10 as compensation for their time. Participants ranged from 19 to 41 years in age (M = 23.83, SD = 6.98). 

Three additional participants were unable to complete the study due to motion sickness, and a fourth additional participant appeared to have misunderstood the instructions and was excluded from the analysis. The experiment was approved by the Office of Research Ethics at Simon Fraser University and conducted in accordance with the WMA Declaration of Helsinki. All participants gave their written informed consent prior to the experiment. 

Stimuli and apparatus

Biomechanical stimuli

Throughout the experiment, participants were seated comfortably on a stationary chair mounted centrally above a circular treadmill, as illustrated in Figure 1. For the biomechanical-only and bimodal conditions, participants were asked to actively step along with the rotating platform, as illustrated in Figure 1. Note that participants never rotated physically in any of the conditions. 

Visual stimuli

The rotating visual stimulus consisted of a nonstereoscopic panorama scene of the central marketplace in Tübingen, Germany (see Figure 2). The scene was rendered in real time at 60 frames/s using a WorldViz Vizard and displayed on a position-tracked NVIS SX111 HMD. The HMD provided a resolution of 1280 × 1024 pixels per eye and a physical field of view of 102° horizontally × 64° vertically (with 50% overlap between eyes). This matched the simulated field of view used for rendering the virtual scene. Viewpoint orientation was actively updated by a Polhemus Liberty tracking system. 

Figure 2

 

360° panoramic round shot of the marketplace in Tübingen, Germany, that was used to create a naturalistic visual stimulus for visually inducing circular vection.

For each trial, visual or biomechanical stimuli rotated in unison at 30°/s for 45 s. There was an initial smooth acceleration and final deceleration of the platform to facilitate stepping along and to reduce potential occurrences of motion sickness. 

Interaction

Participants used a handheld wireless joystick (Logitech Freedom 2.4) to indicate the onset of vection via button press. Following each trial, participants also used the joystick to rate how intense and how convincing the perceived self-rotation was, using a horizontal scale displayed on the HMD. To exclude potentially interfering ambient sound from the lab, participants wore active noise-canceling headphones (Audiotechnica ATH-ANC7) that displayed a mixed-river masking sound at moderate volume. In addition, the headphones were used for providing computer-generated verbal instructions. 

Procedure

Following a written confirmation of informed consent, participants received written and oral descriptions of the experimental procedure and self-motion illusions. After the instruction phase, participants completed four practice trials to familiarize themselves with the procedure and vection experience. The order of conditions for this initial practice block was fixed, starting with bimodal vection, followed by biomechanical-only, visual-only, and bimodal again. ¹ Following each practice trial, the experimenter confirmed the participants' understanding of the instructions and their experience of illusory self-rotation. All participants reported vection. 

To reduce motion sickness and avoid carryover effects between experimental conditions, participants were given a mandatory break upon completion of each block of four trials. Participants were asked to take off the HMD and headphones and step down from the circular treadmill, and the experimenter assessed their general state of nausea to ensure that they were able to continue the experiment. After participants returned to their original position on the circular treadmill, the HMD and headphones were repositioned and viewpoint tracking was recalibrated. Participants were then instructed to prepare for the specific rotation condition prior to each block. Before the biomechanical-only condition, participants received screen text and spoken instructions to close their eyes. For the visual-only and bimodal conditions, participants were instructed to keep their eyes open and neither stare at nor fixate on any particular point of the visual scene during rotation, but instead to observe the visual stimulus in a natural and relaxed manner. Even though adding a fixation point is known to enhance vection (Becker, Raab, et al., 2002; Fushiki, Takata, & Watanabe, 2000), we decided to use natural viewing conditions, as they more closely match typical VR applications and natural situations. Throughout the experiment, the surrounding room was darkened to avoid any potential interference from ambient light. 

After those instructions, participants performed each block of four trials without outside interruption. Another break followed, and the process was repeated until the conclusion of the experiment. Participants indicated vection onset by pressing a designated button on the joystick they were holding. Note that each trial continued for the full 45 s regardless of when the participants indicated vection onset. After each trial, two introspective questions were presented through the HMD. The first question visually and textually asked, “How intense was your sensation of self-motion?,” while the second question asked, “How convincing was your sensation of actually rotating?” Both questions were answered with the joystick operating a sliding scale centered on 50%, with the lowest value at 0% and the highest at 100%. Note that we used introspective response measures for assessing vection in this study, as we were interested in investigating participants' experience of self-motion, a phenomenon that is fundamentally introspective in nature. Moreover, there are to date no reliable behavioral or physiological indicators for vection, making introspective measures the de facto gold standard in vection research. To avoid potential experimenter bias, per-trial instructions were scripted and presented by the computer instead of the human experimenter. Upon completion of the main experiment, participants stepped off the treadmill and were debriefed, thanked, and compensated for their participation. During debriefing, they were asked to verbally rate the vection intensity of the three conditions on the same 0%–100% scale as during the experiment. 

Results

Probability of not experiencing vection

Overall, participants reported compelling vection in most trials. There were, however, a few trials where vection was not reported. The first analysis investigated whether this failure to report vection was systematically related to the different stimulus conditions. 

For each participant, we estimated the probability of not experiencing vection. This estimate was based on a binomial count of trials (out of the four trials corresponding to a particular treatment) in which participants did not report vection. The advantage of this parametric approach over alternative nonparametric models is a greater statistical power to detect differences while requiring only an assumption that a count of successes and failures would follow a binomial distribution. 

In our model we estimated the probability of not experiencing vection using the count of trials in which participants failed to report vection onset for each Participant × Treatment combination. Table 1 shows estimated probabilities of not experiencing vection for each experimental treatment. A binomial mixed-effects logistic model explored how these counts were influenced by period and treatment as fixed-effect predictors and participant as a random effect. This model was computed in SAS 9.3 using the GLIMMIX procedure. The type III tests for fixed effects suggest that neither period, F(2, 32) = 0.41, p = 0.67, nor treatment, F(2, 32) = 0.41, p = 0.67, had a significant effect on the probability of not experiencing vection. In other words, our model shows no evidence that participants' likelihood of experiencing vection or not depended on treatment, even though this finding does not guarantee that such systematic differences do not exist. 

Table 1

 

Mean probabilities of failing to achieve the vection illusion, estimated based on a logistic model using SAS.

Treatment	Probability	95% confidence interval
Biomechanical-only	0.043	[0.013, 0.139]
Bimodal	0.020	[0.004, 0.097]
Visual-only	0.033	[0.008, 0.123]

At the same time, odds ratios shown in Table 2 indicate that effect sizes representing potential differences in the probability of not experiencing vection between experimental treatments were relatively small. Based on the small estimated effect sizes, we conclude that such systematic effects that may exist were unlikely to introduce a substantial bias to the primary measures of vection. Therefore, we treated trials where participants did not report vection as occurring at random and excluded these trials from all subsequent analyses. 

Table 2

 

Odds ratios for failure to achieve vection.

For	Relative to	Odds ratio
Biomechanical-only	Bimodal	2.28
Visual-only	Bimodal	1.73
Biomechanical-only	Visual-only	1.32

Primary vection measures

To remove some of the random within-subject variability, the remaining analyses used means of vection onset time, intensity, and convincingness computed over the four replications in each Participant × Treatment pair. In cases where participants did not report vection in one or more trials, the means were computed over the remaining replications per condition. 

For each of the three measures, the effects of experimental treatment were analyzed using a mixed-effects model with period and treatment as fixed-effect predictors and participant as a random effect. To reduce possible carryover effects, we included breaks between blocks of four trials and pauses between trials as washout measures. We can thus assume in the statistical models that there were no significant carryover effects. These models were fitted in the JMP 10 statistical software package using a restricted maximum likelihood method. 

Vection onset latency

Vection onset latencies showed considerable variability and ranged from 1.7 to 47 s. For the bimodal condition, 20% of trials showed vection onset latencies below 5 s, compared to only 12% for the visual-only condition and 3% for the biomechanical-only condition. Similarly, median vection onset latencies were lowest for the bimodal condition (9.8 s), followed by the visual-only condition (9.9 s) and the biomechanical-only condition (13.9 s). Mean vection onset latencies had a similar pattern, as illustrated in Figure 3 (left). 

Figure 3

 

Mean vection onset latencies (left), vection intensity ratings (middle), and convincingness ratings (right), with 95% confidence intervals.

The model for vection onset latency revealed a significant effect of treatment, F(2, 32) = 5.34, p = 0.01, η² = 0.24, indicating that 24% of the variability in the data was accounted for by the different vection conditions. The effect of period, though, did not reach statistical significance, F(2, 32) = 0.92, p = 0.41, η² = 0.1. Almost half of the overall variance in vection onset latency (49.9%) was accounted for by participant (between-subject) variance. 

Planned contrasts showed that onset latencies for bimodal stimulation were significantly lower than for biomechanical-only stimulation, by 5.82 s, 95% confidence interval (CI) of [1.43, 10.22], t(32) = −3.26, p = 0.0026. Bimodal stimulation yielded somewhat earlier mean vection onset compared to visual-only stimulation, with a difference of 2.55 s, 95% CI of [−1.09, 6.19], but this difference did not reach significance, t(32) = 1.43, p = 0.164. 

Vection intensity ratings

As illustrated in Figure 3 (middle), vection was rated as most intense in the bimodal condition, followed by the biomechanical-only and the visual-only condition. This was confirmed by the model for vection intensity ratings, which revealed a significant effect of treatment, F(2, 32) = 4.63, p = 0.017, η² = 0.2. The effect of period was not statistically significant, F(2, 32) = 2.45, p = 0.102, η² = 0.04. The analysis of variance component estimates showed that effect of participant (between-subject variance) accounted for 45.5% of overall variance. Planned contrasts showed that vection intensity ratings for bimodal stimulation were higher than for biomechanical-only stimulation by 9.0% on a 0%–100% scale, 95% CI of [0.8%, 17.2%], t(32) = 2.24, p = 0.032. Similarly, vection intensity ratings for bimodal stimulation were 11.7% higher than for the visual-only condition, 95% CI of [3.5%, 19.9%], which was statistically significant, t(32) = 2.9, p = 0.0066. 

Vection convincingness ratings

Overall convincingness of the self-motion illusion was rated as relatively high (>60%) in all conditions (see Figure 3, right). The model for ratings of vection convincingness suggested a trend (p < 0.10) for a main effect of treatment, F(2, 32) = 2.5, p = 0.098, η² = 0.12, and no significant effect for period, F(2, 32) = 1.65, p = 0.208, η² = 0.08. Between-subject variability accounted for 54.9% of overall variance. 

Postexperimental vection intensity ratings

During the postexperimental debriefing, participants were asked to rate the overall vection intensity of the three vection conditions (visual, biomechanical, and bimodal), on a scale from 0% (It did not feel at all like rotating) to 100% (I really felt like I was moving). Similar to the intensity ratings collected during the experiment, mean postexperimental ratings showed overall high vection intensity (see Figure 4, left) and a tendency for more intense vection in the bimodal condition. However, this trend did not reach significance, F(2, 32) = 2.6, p = 0.093, η² = 0.09. 

Figure 4

 

Left: Mean vection intensity ratings from the postexperimental debriefing with 95% confidence intervals. Right: Histogram of participants' reports of how much they played 3-D computer or console games on average per week.

Participant data: 3-D computer game play

During the debriefing, participants were asked to report how much they played 3-D computer or console games on average, as we were interested in investigating potential correlation to their susceptibility to vection. While seven out of the 18 participants (39%) did not play 3-D computer games regularly, the remaining participants played on average between 1 and 21 h per week, as illustrated in Figure 4 (right). 

Correlation analysis revealed that more frequent 3-D game play correlated with lower vection convincingness ratings in the visual-only vection condition, r = −0.488, p = 0.040, and the bimodal vection condition, r = −0.560, p = 0.015. Although there was also a negative correlation in the biomechanical vection condition, it was not reliable, r = −0.311, p = 0.208. Neither vection onset latencies nor vection intensity ratings showed significant correlations with 3-D game play in any of the three vection conditions, all |r|s < 0.28, ps > 0.25. That is, although experienced computer gamers seemed to be less easily convinced of a visually induced self-motion illusion, we found no evidence that they experienced it less intensely or with later vection onset compared to nongamers. 

Correlation between vection intensity ratings during and after the experiment

To investigate whether participants' retrospective assessment of their perceived self-motion intensity matches their online vection ratings and can serve as a reliable measure of their perceived vection intensity during the experiment, we conducted for each of the three stimulus conditions pair-wise correlations between participants' per-trial vection intensity ratings during the experiment and the overall vection intensity rating they provided after the experiment in the postexperimental debriefing. Reliable positive correlations were present in all three conditions (visual-only: r = 0.654, p = 0.0032; bimodal: r = 0.578, p = 0.012; and biomechanical: r = 0.520, p = 0.027). This indicates fairly consistent scale use and reasonable overall reliability of postexperimental vection ratings, and suggest that participants did not drastically adjust or change their vection experience ratings after having experienced all three vection conditions due to, e.g., carryover effects between conditions or overall learning effects. However, it is conceivable that performing ratings during the experiment might have affected the postexperimental ratings, and further research is needed to investigate the quality and reliability of postexperimental ratings by themselves. 

Discussion

The current study was designed to assess whether the embodied sensation of self-motion (circular vection) induced by visually simulated self-rotation in a naturalistic scene presented via wide-FOV HMD could be enhanced by matching concomitant walking motions on a circular treadmill. To this end, we compared the intensity, convincingness, and onset latency of the self-motion illusion in three stimulus conditions for stationary seated observers: visual-only, where participants viewed a simulated self-rotation via HMD; biomechanical-only, where participants stepped along the rotating floor in complete darkness, thus mimicking the walking motions of actual self-rotation; and bimodal, where visual and walking motions were matched to simulate an actual self-rotation. 

Visual and biomechanical cues were overall similarly effective in inducing vection, and provided strong and compelling vection in more than 95% of all trials. Despite these strong overall vection experiences, combining visual and biomechanical cues further enhanced vection significantly and resulted in the earliest vection onset and the most intense sensation of self-motion of the three conditions. 

To the best of our knowledge, this is the first study to demonstrate that visually induced illusions of self-rotation in VR can be reliably induced in a head-mounted display and then further enhanced by matching walking motions in otherwise stationary observers. This extends previous findings that combining different sensory modalities can enhance rotational self-motion illusions, as has been shown for visual-auditory vection (Riecke et al., 2009; Väljamäe, 2009) and auditory-biomechanical vection (Riecke et al., 2011). It thus helps to close the gap in our theoretical understanding of how different sensory modalities can be used to more reliably induce vection and opens the door for future computational modeling studies. Such research could shed further light on how the different stimulus parameters contribute and quantify in more detail the relative contributions of visual, biomechanical, and other cues. 

The following sections discuss key results related to inducing vection illusions in HMDs, multisensory contributions to vection, and an outlook for practical applications of vection research in VR. We also discuss a curious finding related to the observed individual differences in perception of self-motion illusions between participants with different exposures to 3-D gaming. 

Challenges of inducing vection in HMDs

The current study might be the first to demonstrate that visual circular vection can be reliably elicited using a head-mounted display. The vast majority of prior circular vection studies used projection screens or optokinetic drums instead of HMDs. In fact, the experience of the authors and other vection researchers suggests that HMDs with limited FOVs (say, 50° × 40° vs. 102° × 64° in the current study) do not reliably induce compelling circular vection in VR. Although under carefully controlled lab conditions, vection has been induced for FOVs as small as 7.5° (Andersen, 1986), increasing the FOV of the moving visual stimulus generally enhances vection in all dependent measures, to a point where stimulating the users' entire visual field can result in self-motion percepts that are no longer distinguishable from actual self-motion (Berthoz et al., 1975; Brandt et al., 1973; Dichgans & Brandt, 1978; Held et al., 1975). The importance of a sufficiently large FOV for reliably inducing compelling self-motion sensations is exemplified by findings from Nakamura (2008) indicating that vection strength increases linearly with increasing FOV. 

Self-motion illusions typically do not start right with the onset of visual motion, but only after a certain vection onset latency, which can range from a few seconds to more than 20 s (Dichgans & Brandt, 1978; Hettinger, 2002; Riecke, 2011). Given how important display factors are for inducing vection (Dichgans & Brandt, 1978; Hettinger, 2002; Palmisano et al., 2011; Riecke, 2011), a direct comparison with earlier studies is difficult. Nevertheless, onset latencies for the visual-only condition are within the typical range observed when participants are not provided with full-field stimulation (for reviews, see Andersen, 1986; Dichgans & Brandt, 1978; Hettinger, 2002; Mergner & Becker, 1990; Riecke, 2011; Warren & Wertheim, 1990). Participants in our study reported strong and compelling sensations of self-motion in almost all trials, with a typical vection onset delay of 15.3 s in the visual-only condition and 12.7 s in the bimodal condition. Early vection onset was rare: In only 20% of trials was bimodal vection perceived within 5 s, and only for 6% of trials did it occur within 3 s. Therefore, despite the relatively wide FOV of our HMD, onset latencies were still much longer than typically observed for full-field stimulation in an optokinetic drum, where participants on average perceive circular vection within only 3–4 s (Brandt et al., 1973) and for some participants the onset occurs after as little as 1 s (Dichgans & Brandt, 1978). Together, this suggests that immersive simulations presented on contemporary wide-FOV HMDs are perceptually still far away from reaching the full potential of real-world, full-field stimulation. 

One concern with the current study was the occurrence of motion or simulator sickness in three participants excluded from further analysis (i.e., in 14% of all participants tested). While it is not uncommon to experience simulator sickness in wide-FOV HMDs like the one used in the current study, even when participants do not experience vection, this is an issue worth considering. This is especially true in the context of VR and motion simulation, where vection can be a cost-effective means of enhancing the overall realism and convincingness. There have been several studies showing that vection can indeed correlate with undesirable side effects like simulator sickness or motion aftereffects (Hettinger, 2002; Hettinger, Berbaum, Kennedy, Dunlap, & Nolan, 1990; Kennedy et al., 2003; Palmisano, Bonato, Bubka, & Folder, 2007). It remains, however, an open research question whether or how vection might also causally affect simulator sickness, which can occur without any experience of vection (Ji, So, & Cheung, 2009). Moreover, whereas simulator sickness tends to increase for larger visual-vestibular cue conflicts, vection tends to decrease (Kennedy et al., 2003; Palmisano et al., 2007). Nevertheless, as more wide-FOV displays capable of inducing strong vection become affordable and available, undesirable side effects like simulator sickness will likely also increase, and they should be carefully considered in both research and applications (Hale & Stanney, 2014). 

Multisensory contributions to vection and outlook for applications in VR

In the current study, we found no significant multisensory effects of combining visual and biomechanical stimulation on vection onset latency. While on average, participants in the bimodal condition experienced vection significantly earlier than in the biomechanical-only condition, we did not see a significant improvement relative to the visual-only condition. The reported onset times of biomechanically induced self-motion illusions are typically much higher than the extremely low latencies for visually induced vection initiated by optokinetic drums discussed in the previous section. For example, Lackner and DiZio (1984) observed onset latencies of 21.8 s, and Bruggeman and colleagues (2009) reported onset latencies of about 20 s. This onset latency is similar to the vestibular time constant of 15–20 s, which is the time typically needed for vestibular signals to decay and the canals to stabilize following a prior acceleration signal (Brandt, Dichgans, & Büchele, 1974; Young, 1984). The mean onset latency of 18.5 s observed for biomechanical-only vection in our experiment is in overall agreement with prior research and was also longer than the comparable measure for the visual-only condition. These findings support the dominance of visual over biomechanical cues with respect to vection onset delay. 

At the same time, we found a significant improvement in the reported intensity ratings of vection for the bimodal condition compared to both the visual-only and biomechanical-only conditions. These findings suggest that while the onset of self-motion illusions may not have occurred any sooner, the combination of visual and biomechanical stimulation led to a measurable increase in the quality of the illusion. Given the limitation of the vection-inducing potential of current wide-FOV HMDs, one might expect that this effect will be even more pronounced for displays capable of stimulating a full visual field of view. 

From an applied perspective, our findings indicate that locomotion interfaces combining visual and biomechanical cues might be an interesting target for future development. It is noteworthy that biomechanical vection can be induced by both stepping along a motor-driven rotating platform (as in our study) and actively pedaling a free-moving platform (Lackner & DiZio, 1984). While our experiment relied on a relatively complex mechanical circular treadmill to deliver consistent biomechanical feedback to all participants, such a complex setup is not necessary to gain the vection-inducing benefits of biomechanical feedback. Removing the need for motor-driven platforms by having participants actively propel a simple free-wheeling platform reduces technical complexity and the cost of incorporating biomechanical vection-inducing cues into motion simulations. We are currently exploring this idea within an active input paradigm for rotational motion simulation. 

Compared to walking on circular treadmills, which has repeatedly been shown to induce compelling self-motion illusions in the absence of any supporting visual cues (Bles, 1981; Bles & Kapteyn, 1977; Bruggeman et al., 2009; DiZio & Lackner, 2002; Lackner & DiZio, 1984, 1988; Riecke et al., 2011) or even with conflicting visual cues (Lackner & DiZio, 1988), walking on linear treadmills by itself does not seem to be sufficient to induce any reliable sensation of forward movement unless combined with visual cues (Durgin et al., 2005; Riecke & Schulte-Pelkum, 2013). Moreover, recent studies indicate that visually induced sensations of forward movement (“vection in depth”) can be significantly impaired by concomitant walking on a linear treadmill, even if optic flow and walking speeds are closely matched (Ash et al., 2013; Kitazaki et al., 2010; Onimaru et al., 2010). That is, participants experienced stronger sensations of forward motion when viewing a visually simulated forward motion while standing still as compared to walking forward on a linear treadmill at a speed matching the visual simulation of comfortable walking speeds (4 and 5 km/h). This reduction in self-motion perception during forward walking on a linear treadmill has been replicated for different natural walking speeds and visual stimuli of different vection-inducing strengths (Ash et al., 2013), and occurs even when participants are walking in place as compared to standing still (Ash, Palmisano, & Allison, 2012). Given these previous results, our finding that visually induced illusions of self-rotation can be significantly enhanced by stepping along a circular treadmill seems quite remarkable. Taken together, these findings challenge the predominant focus on linear walking interfaces for VR locomotion, and suggest that investigating circular or curvilinear walking might be a promising direction for future research and development. In particular, future research could investigate how the current results might or might not generalize to a wider range of visual stimuli, motion trajectories, walking patterns, and participant populations. For example, for visually induced vection, curvilinear trajectories tend to be more effective in inducing vection when compared with straight (forward or backward) paths without any rotational component (Riecke & Feuereissen, 2012; Trutoiu et al., 2009). If a similar benefit of adding rotational components were found for biomechanical cues, this would be of significant interest both for motion simulation applications and our understanding of cue integration. 

Individual differences in perceiving self-motion illusions

Somewhat surprisingly, we also found negative correlations between vection convincingness ratings and self-reported frequency of playing 3-D computer and video games. We hypothesize that gamers might be more accustomed to and perhaps more desensitized to the influence of visual components of self-motion simulations through complex virtual worlds. This, in turn, could lead them to be unimpressed or unconvinced by the relatively banal visual stimuli displayed during this experiment, thus causing them to judge their self-motion experience as less convincing. 

Interestingly, only the convincingness of vection correlated with participants' gaming experience—not vection onset latency or intensity. This suggests that vection convincingness ratings might be more susceptible to higher level cognitive influences as compared to vection intensity or onset latency measures, which were not significantly affected by gaming experience. These findings parallel and extend observations by Wright, DiZio, and Lackner (2006), where manipulating higher level cognitive factors (whether or not the simulated scene matched the physical environment, which might be related to spatial presence) affected only the compellingness ratings of perceived self-motion, not onset latency or amplitude of perceived self-motion. Similarly, Riecke and colleagues have reported that the convincingness of self-motion illusions in VR was more closely related to users' sense of spatial presence in the virtual environment, whereas involvement in the simulation was more closely related to vection onset latency (Riecke & Schulte-Pelkum, 2014; Riecke et al., 2006). 

Our present findings highlight the relevance of higher level cognitive factors, such as prior experience and spatial presence, for creating compelling self-motion simulations in VR, and suggest that such factors should be taken into consideration when designing VR simulations (see also discussions in Riecke, 2011; Riecke & Schulte-Pelkum, 2013, 2014). Future research combining additional qualitative with quantitative measures and a larger, more diverse participant sample is needed to fully test our hypothesis. If corroborated, it could support the argument that extended experience with gaming or other immersive stimuli can have long-term effects on the effectiveness of visual stimulation, which would have both theoretical and applied implications. 

Conclusion

In combination with other simple and user-powered motion-cueing techniques (Riecke, 2006; Riecke & Feuereissen, 2012), vection can potentially be used to significantly enhance self-motion perception in VR using inexpensive locomotion interfaces that do not require actual physical walking or complex motion simulators. As self-motion illusions have been shown to facilitate spatial orientation (Riecke et al., 2012) and correlate with enhanced presence and involvement in the simulation (Riecke & Schulte-Pelkum, 2014; Riecke et al., 2006; Wright et al., 2006), improving self-motion perception in VR might also help to improve overall user experience and the overall effectiveness of a VR simulation. A multisensory repertoire for inducing self-motion illusions including auditory, visual, biomechanical, and vibrational cues, as well as higher level cognitive contributions (Riecke, 2011; Riecke & Schulte-Pelkum, 2013; Wright et al., 2006), may also be useful in enhancing existing locomotion interfaces, as it provides additional flexibility. 

Our study demonstrates a benefit when combining biomechanical and visual cues for inducing rotational self-motion illusions in an HMD-based virtual reality system. We suggest that future rotational locomotion interfaces may benefit from adding multisensory stimulation to create a compelling sensation of self-rotation. Future investigation into potential multimodal contribution to self-motion illusions will expand our understanding of human self-motion perception and may contribute to improved locomotion interfaces in VR. 

Acknowledgments

Support was provided by the Natural Sciences and Engineering Research Council of Canada and Simon Fraser University. 

Commercial relationships: none. 

Corresponding author: Bernhard E. Riecke. 

Email: [e-mail]. 

Address: SIAT, Simon Fraser University, Surrey, BC, Canada. 

Footnotes

¹  This fixed order ensured that participants experienced the same practice stimuli, and that they experienced the bimodal condition (which was hypothesized to yield the strongest vection) again at the end of the practice phase to properly anchor their vection ratings before the subsequent main experiment. We are not aware of any research showing that such a fixed order during practice trials could systematically affect the subsequent main experiment.