**We introduce OpenEyeSim, a detailed three-dimensional biomechanical model of the human extraocular eye muscles including a visualization of a virtual environment. The main purpose of OpenEyeSim is to serve as a platform for developing models of the joint learning of visual representations and eye-movement control in the perception–action cycle. The architecture and dynamic muscle properties are based on measurements of the human oculomotor system. We show that our model can reproduce different types of eye movements. Additionally, our model is able to calculate metabolic costs of eye movements. It is also able to simulate different eye disorders, such as different forms of strabismus. We propose OpenEyeSim as a platform for studying many of the complexities of oculomotor control and learning during normal and abnormal visual development.**

*movements*, models using simplified properties of EOMs have been developed (Quaia & Optican, 1998; Raphan, 1998). These models do not take into account the anatomical variations of different EOMs, such as muscle length, cross-sectional areas, and muscle forces. In some previously developed models, muscle forces were assumed to be proportional to the innervation, whereas real muscle forces are complex functions that depend on muscle length, velocity, and innervation. To our knowledge, only the model presented by Wei, Sueda, and Pai (2010) incorporates proper muscle dynamics and produces realistic eye movements. However, all these models only cover the biomechanical component, neglecting any visual processing.

- Realistic EOM force dynamics, which to our knowledge have only been presented in one previous 3-D model of eye muscles (Wei et al., 2010)
- Muscle pulleys—extraocular connective tissues which stabilize muscle paths
- Realistic muscle paths, based on measurements in humans
- Visualization of a virtual environment, which is crucial for simulations of closed-loop visuomotor control and learning

*a*∈ [0, 1] is the muscle activation (0 = fully relaxed muscle, 1 = fully innervated muscle),

*f*

_{FL}is the force–length relationship,

*f*

_{FV}is the force–velocity relationship,

*l*is the muscle length, and

*v*is the contraction speed. The passive elastic force

*F*

_{PE}is calculated as a function of muscle length: where

*l*is the muscle length and

*f*

_{p}is the passive force–length relationship (for a detailed explanation, see Supplementary Appendix A).

*P*has been initially measured by Robinson and can be represented as where

*e*is the degree of rotation and

*k*and

_{p}*k*are coefficients. Robinson's measurements (Robinson et al., 1969) found

_{c}*k*= 0.48 g/° and

_{p}*k*= 1.56 g/°

_{c}^{3}. However, later studies by Collins (Robinson, 1981) showed that the proper value for

*k*is 0.32 g/°. These forces try to keep an eyeball in its primary position and serve its stabilization. In our model we used the built-in functionality of OpenSim; we were therefore able to include only the first term in our simulations. However, neglecting the cubic component should not result in a big change in the model dynamics. For example, at 20° eye rotation the contribution of the cubic component is around 11% of total elasticity force.

_{p}*A*is the activation heat rate of muscle

_{i}*i*,

*M*is the maintenance heat rate,

_{i}*S*is the shortening heat rate, and

_{i}*W*is the mechanical heat rate (see Supplementary Appendix B for details). A calculation of these values requires additional parameters—muscle masses and ratios of slow-to fast-twitch fibers. Values for the ratio of slow- to fast-twitch fibers were roughly estimated based on histological studies on EOMs (Wasicky et al., 2000) and are 15.1%/84.9% for central muscle parts and 14.3%/85.7% for peripheral muscle parts. These studies showed that nearly 85% of muscle fibers found in EOMs have the same characteristics as fiber types found in skeletal muscles. Another 15% of fibers have different specific structures, which are not present in skeletal muscles. Because of the lack of information on these fiber types, we neglect their contribution (which has been shown to be present in some saccades; Porter, Baker, Ragusa, & Brueckner, 1995) and simply calculate metabolic costs based on the ratio of slow- to fast-twitch fibers of 85% of total volume. Figure 4 shows an example of such calculations of metabolic costs. Different lateral and medial rectus muscle activations that lead to the same static eye positions are indicated by white lines. The energy consumption for using such muscle activations for 1 s is presented by different colors. This figure reveals the redundancy in oculomotor control: Different muscle activations lead to the same eye position.

_{i}**p͐**

*in the search space. Such a point*

_{i}**p͐**

*is sampled from a multivariate Gaussian distribution 𝒩(*

_{i}**p͐**(

*t*); Σ(

*t*)), where

**p͐**is the mean, Σ is the covariance, and

*t*is the iteration number. The sampling space is modified over iterations based on successful samples. The algorithm finds a solution whenever a sample reaches a predefined acceptance threshold.

*n*samples are drawn: where each sample

**p͐**is used as an input to the forward-dynamics module of the OpenSim simulator. The muscle activations are held constant until the eyeball has settled into the corresponding gaze position. The produced rotations, as well as muscle activations, are used to calculate a reward: where

_{i}**p͐**;

_{i}*j*for sampled solution

*i*; and

*s*is a scaling factor. This reward function penalizes high activation levels while favoring eye rotations close to the desired. We have chosen a value of

*s*= 0.5. As shown in Figure 4, solutions based on this reward function correspond to low metabolic costs. The samples

**p͐**, their corresponding rewards

_{i}*q*, and the search parameters

_{i}**p͐**(

*t*) and Σ(

*t*) are used to generate new search parameters

**p͐**(

*t*+ 1) and Σ(

*t*+ 1) to produce the next generation of samples.

- The test subject wears glasses with green–red filters with the red filter in front of the right eye, which is called the fixing eye. Then the patient is given a green-light pointer while the examiner uses a red-light pointer.
- The examiner places the red-light dot into different positions on the Hess screen covering the main working area of gaze positions. The tested subject is asked to bring the green-light dot (seen by the following eye) over the red-light dot. In the normal case, these dots should overlap.
- The same procedure is repeated with changed filters.

- Calculate activation levels for eye muscles under normal conditions for different points on the Hess–Lancaster diagram. Such calculations are performed using the black-box optimization algorithm presented in Methods. The obtained muscle activation values for the nine points on the Hess–Lancaster diagram are given in Supplementary Appendix C.
- Scale the maximum activation levels of affected muscles to reduce calculated forces at different gaze positions.
- Produce forward-dynamics simulations with reduced activation signals to calculate the altered eye positions.

*Advances in Optics and Photonics*, 6 (3), 340–367.

*Vision Research*, 17 (9), 1037–1044.

*Mathematical Biosciences*, 20 (3), 213–238.

*The control of eye movements*(pp. 283–325 ). New York: Academic Press.

*IEEE Transactions on Biomedical Engineering*, 54 (11), 1940–1950.

*Visual optics*. London: Hatton Press.

*Neuron*, 47 (2), 281–293.

*Towards a new evolutionary computation*(pp. 75–102). Berlin, Germany: Springer.

*Annals of the New York Academy of Sciences*, 1039 (1), 9–14.

*Proceedings of the Royal Society of London B: Biological Sciences*, 126 (843), 136–195.

*Strabismus*. Stuttgart, Germany: Georg Thieme Verlag.

*The Journal of Neuroscience*, 26 (10), 2732–2737.

*SIAM Journal on Optimization*, 9 (1), 112–147.

*Journal of Biomechanical Engineering*, 135 (2), 021005.

*Vision Research*, 29 (2), 223–240.

*Computers and Biomedical Research*, 17 (5), 436–470.

*Survey of Ophthalmology*, 39 (6), 451–484.

*Journal of Neurophysiology*, 79 (6), 3197–3215.

*Frontiers in Integrative Neuroscience*8, 71.

*Journal of Neurophysiology*, 79 (5), 2653–2667.

*The Journal of Physiology*, 174 (2), 245–264.

*Experimental Neurology*, 22 (1), 130–132.

*Models of oculomotor behavior and control*(pp. 21–41 ). Boca Raton, FL: CRC Press.

*Journal of Applied Physiology*, 26 (5), 548–553.

*Principles of neural science*(Vol. 4, pp. 1227–1246 ). New York: McGraw-Hill.

*Strabismus*, 4 (1), 25–40.

*Journal of Neurophysiology*, 82 (5), 2612–2632.

*Applied Optics*, 31 (19), 3594–3600.

*Science*, 257 (5074), 1279–1281.

*Computer Methods in Biomechanics and Biomedical Engineering*, 6 (2), 99–111.

*Ber. Verh. Sächsische Gesellschaft der Wissenschaften*, 21–28.

*Progress in Biophysics and Molecular Biology*, 103 (2), 273–283.