Abstract
The experimental control granted by virtual reality (VR) allows investigation of complex behaviors involving naturalistic stimuli. VR engines provide direct access to the images viewed by subjects, facilitating analyses of image properties that are often difficult to extract from real scenes. For example, current computer vision algorithms of optic flow are often inaccurate in complex scenes with significant depth variations. We have developed a prototypical model of the eye to allow for real-time recording of visual stimuli projected onto the retina from the virtual environment. The Panfili Functional Eye (PFE) model uses real-time ray tracing to compute the optic flow stimuli rendered by the VR engine. The PFE uses a pinhole model of the eye with refraction, with input parameters designating resolution, field of view, and movement method. An array of virtual photoreceptors is generated along the surface of a virtual retina. The virtual photoreceptors cast rays through the pupil, where refraction is applied using Snell’s Law. These rays are then cast out to the virtual environment, returning information such as world position, normal of the vector, and coordinates on the UV/Lightmap of the object. The primary goal of the model is to describe the geometric projection of the virtual environment onto the retina in real-time. High-fidelity and low-latency retinal modeling has not been previously possible due to the technical limitations of ray tracing. The model performs as much as 150 times faster than comparable methods, a metric which should increase exponentially with the use of parallel processing. The PFE is modular to allow for the incorporation of more complex optical models, simulating eye conditions, and the analysis of other visual features. Our current application uses the model to compute optic flow patterns of experimental stimuli contingent on direction of gaze while subjects walk freely in a virtual environment.