We present a novel display that spatio-temporally filters a movie as a function of gaze. For each pixel, filter parameters can be specified in retinal coordinates. Perry and Geisler were the first to simulate arbitrary visual fields by creating a spatial Gaussian multi-resolution pyramid; at each pixel of the output image, two adjacent pyramid levels were interpolated to obtain a lowpass-filtered version of the input pixel.
We now extend this idea to the temporal domain. Furthermore, our display is based on a Laplacian instead of a Gaussian pyramid, so that not only a cutoff frequency can be specified for each pixel, but a full set of weights for individual spatio-temporal frequency bands.
Because of the high computational cost of upsampling all levels of the pyramid to full spatio-temporal resolution in each frame, we implemented our algorithm on the Graphics Processing Unit using the Cg shader language. We achieve real-time performance ([[gt]]30 fps) on HDTV (1280×720 pixels) videos.
We aim to use our gaze-contingent display to selectively suppress and enhance spatio-temporal content as a function of current gaze. By changing the saliency distribution of a movie in real time, we intend to guide the observer's gaze such as to improve visual communication (see http://www.gazecom.eu).
Perry, J., Geisler, W. (2002). Gaze-contingent real-time simulation of arbitrary visual fields. In Human Vision and Electronic Imaging: Proceedings of SPIE. 4662, 57–69.
Böhme, M., Dorr, M., Martinetz, T., Barth, E. (2006). Gaze-contingent temporal filtering of video. In Proceedings of Eye Tracking Research & Applications (ETRA), 109–115.
Our research has received funding from the European Commission within the GazeCom project (IST-C-033816) of the FP6 (
http://www.gazecom.eu).