Purpose: Small step displacements of a large random-dot pattern produce vivid percepts of apparent (phi) motion. Reversing the luminance polarity during the steps results in apparent motion in the opposite direction: reversed phi. We recorded the short-latency ocular following responses to step displacements with and without luminance reversals in humans. Methods: Binocular eye movements were recorded with the scleral search coil technique from the 3 authors and 1 naive subject. Subjects faced a large tangent screen (80×50°) onto which random-dot patterns (dot diameters, 2°; 50% coverage) were back-projected. Mirror galvanometers in the light path were used to apply horizontal step displacements (range, 0.1–4.8°), during which the screen was blanked for 14ms, shortly (50ms) after a centering saccade (to take advantage of post-saccadic enhancement). On half of the trials, luminance polarity was reversed during the steps by introducing a second image, which was a photographic negative of the pattern, using a second projector. Results: Step displacements without a change in luminance elicited small (<2°/s) but consistent ocular following responses at ultra-short latency (∼75ms) in the direction of the displacements. Tuning curves describing the relationship between the change in eye position (over the time window, 80–160ms) and the magnitude of the step resembled the derivative of a Gaussian and were well fit by Gabor functions, peaking with steps of ∼0.4° and reaching asymptote at ∼1.8°. Reversing the luminance during the steps elicited smaller (by ∼50%) tracking responses (latency: ∼75ms) in the direction opposite to the displacement steps; spatial parameters of the best-fit Gabor functions were little changed except that the cosine term was phase shifted by ∼180°. Conclusions: Initial ocular following is mediated, at least in part, by first-order (luminance) motion-energy detectors.