We recorded horizontal ocular-following responses to pairs of superimposed vertical sine wave gratings moving in opposite directions in human subjects. This configuration elicits a nonlinear interaction: when the relative contrast of the gratings is changed, the response transitions abruptly between the responses elicited by either grating alone. We explore this interaction in pairs of gratings that differ in spatial and temporal frequency and show that all cases can be described as a weighted sum of the responses to each grating presented alone, where the weights are a nonlinear function of stimulus contrast: a nonlinear weighed summation model. The weights depended on the spatial and temporal frequency of the component grating. In many cases the dominant component was not the one that produced the strongest response when presented alone, implying that the neuronal circuits assigning weights precede the stages at which motor responses to visual motion are generated. When the stimulus area was reduced, the relationship between spatial frequency and weight shifted to higher frequencies. This finding may reflect a contribution from surround suppression. The nonlinear interaction is strongest when the two components have similar spatial frequencies, suggesting that the nonlinearity may reflect interactions within single spatial frequency channels. This framework can be extended to stimuli composed of more than two components: our model was able to predict the responses to stimuli composed of three gratings. That this relatively simple model successfully captures the ocular-following responses over a wide range of spatial/temporal frequency and contrast parameters suggests that these interactions reflect a simple mechanism.

^{2}). Visual stimuli—single or two to three vertical sinusoidal luminance gratings (fully overlapping)—were seen through a ∼22° × ∼22° (512 × 512 pixels) rectangular aperture centered directly ahead of the eyes. The stimuli seen by the two eyes were always the same: we used the stereoscope at the outset of the project because we were not sure if we would need binocular manipulations to understand these responses.

^{W}), or produced a response amplitude when presented alone similar to that of the low SF component (high

^{A}). The TF was 18¾ Hz for all gratings. The optimal SF always moved in the same direction as one of the other two components, and these two gratings each had a Michelson contrast of 12%. The component moving in the direction opposite to the other two gratings could have Michelson contrast of ∼7%, 12%, or ∼21%. At the beginning of a trial, the phases of all components were randomized. A single block of trials had 68 randomly interleaved stimuli: 20 single-grating, 24 two-component, and 24 three-component conditions.

^{2}) marking the end of the trial. A new fixation target appeared after a 500 ms intertrial interval, signaling a new trial. The subjects were asked to refrain from blinking or shifting fixation except during the intertrial intervals, but were given no instructions relating to the motion stimuli. If no saccades were detected for the duration of the trial, then the data were stored; otherwise, the trial was aborted and repeated within the same block. Data collection usually occurred over several sessions until each condition had been repeated an adequate number of times to permit good resolution of the responses (through averaging).

^{1}The mean eye velocity was estimated by subtracting position difference measures 10 ms apart (central difference method) and evaluated every millisecond. Response latency was estimated by determining the time after stimulus motion onset when the mean eye velocity first exceeded 0.1°/s. The initial OFRs to a given stimulus were quantified by measuring the changes in the mean horizontal eye position signals—OFR amplitude—over the initial open loop period, that is, over the period up to twice the minimum response latency. This window always commenced at the same time after the stimulus motion onset (“stimulus-locked measures”) and, for a given subject, was the same in all experiments reported in this article: 69–138, 74–148, and 66–132 ms for BMS, EJF, and TH, respectively. Bootstrapping procedures were used for statistical evaluation of the data and to construct 68% confidence intervals of the mean in the figures (these intervals were smaller than the symbol size in many cases and, therefore, not visible on most graphs).

*r*

^{2}= 0.995; red dashed line) by the following equation:

*C*and

_{1}*C*are the contrasts of each component. In this Experiment,

_{2}*R*

_{2}is always negative, because the second grating moves in the opposite direction to the first one. The two free parameters,

*WR*and

*n,*summarize the interaction.

*n*characterizes the steepness the transition between \({R_1}\) and \({R_2}\).

*WR*stands for weight ratio; Equation 1 can be rewritten as

*WR*is equal to 0.81, that is, 0.36 cpd grating is given more weight than 0.22 cpd one and dominates the interaction (quantifying the observations made, while examining mean velocity traces in Figure 2A). Although it is not surprising that fits were very good (three modelled data points and two parameters), this function also provided an excellent account when more contrast levels were used (see Appendix B). It is for this reason that we were able to rely on just three contrast ratios to estimate

*WR*and

*n*for a given grating pair.

*C*, and

_{1}*C*have the same meaning as in Equation 1, and

_{2}*n*and

_{1}*n*are two free parameters. Although Equations 1 and 3 are clearly different, there are situations in which both equations produce identical fits. One of these situations is when the contrast of one component is kept constant, as in the current study. In this case it can be shown that:

_{2}*C*is the contrast used for the fixed contrast component, and

_{1}*n*is fit to the component that changes contrast. A second situation is where

_{2}*WR*

*≈*1, in which case both models give very similar description for the same patterns (i.e., they each provide a good fit to data generated by the other model), even when both contrasts are varied (hence, both models provide equally good descriptions of the data in Sheliga et al. (2006), see below). We ran a series of simulations to identify conditions where the two models were most distinguishable, which we found to be true when both components varied in contrast and the

*WR*was far from 1. We ran one such condition (0.07/0.59 cpd pairing) in subject BMS and found that Equation 1 provided a better description of data than Equation 3 (

*p*= 0.0028, see Figure A1 in Appendix A). Furthermore, the differences between the data and the Equation 1 fit were not statistically significant (

*p*= 0.14). We proceeded, therefore, with using Equation 1 to describe interactions in two-component stimuli.

*WR*and

*n*—depended on component SF. Figure 3A–C shows

*WR*(color-coded small squares; see color bar to the right of panel C) as a function of SF

_{1}and SF

_{2}in a two-component stimulus

^{2}for 3 subjects. Each of five SF pairings contributed two symbols to the plot—symmetrically located across the main diagonal—and their

*WR*values were the inverse of one another. This is because swapping the labels

*SF*and

_{1}*SF*and substituting \(\frac{1}{{WR}}\) is an identity mapping. Extensive pilot experiments in one subject (Appendix B) showed that the value of

_{2}*WR*was well-described by a separable function of SF, relating

*W*to

_{i}*SF*

_{i}:

*μ*

_{W},_{HC}

*,*and σ

_{HC}are three free parameters. Note that the absolute scaling of Equation 6 is unconstrained, because

*W*is only ever used to calculate the weight ratio,

_{i}*WR*= \(\frac{{{W_1}}}{{{W_2}}}\), for a given pair of SFs.

*WRs*of three subjects, shown in Figure 3A–C, were very well fit by Equation 5:

*r*

^{2}= 0.998, 0.995, and 0.996 for subjects BMS, EJF, and TH, respectively. Figure 3D–F plots

*n*(color-coded small squares; see color bar to the right of Figure 3F) as a function of

*SF*

_{1}and

*SF*

_{2}in a two-component stimulus

^{3}for three subjects. Once again, each SF pairing contributed two symbols to the plot, with the same value in symmetrical locations around the main diagonal (the value of

*n*does not depend on the order of SF

_{1}and SF

_{2}). Pilot experiments, which tested much higher number of SF pairings (Appendix C), allowed to suggest a model, describing the dependence of

*n*upon SF

_{1}and SF

_{2}, which was then tested in 3 subjects. There are two notable features in the model. First,

*n*is largest close to the identity line (when the SF ratio is close to 1) and falls off smoothly as the SF ratio increases. We described this with a Gaussian of the

*log*SF ratio:

*n*less than one would lead to low contrast stimuli dominating. Second,

*n*decreases more rapidly with the SF ratio at high and low frequency, with the slowest decline for pairs where the geometric mean SF is ∼0.25 cpd, close to the optimal SF for driving the OFR with single gratings. We capture this feature by allowing the value of sigma in Equation 8 to be a Gaussian function of the log of the product of SFs:

*n*in Equation 1 seems to depend on both the ratio and the product of component SFs:

_{1}, SF

_{2}and

*n*(shown in Figure 3D–F) using four free parameters:

*r*

^{2}= 0.999, 0.976, and 0.993 for subjects BMS, EJF, and TH, respectively.

_{1}TF

_{1}, SF

_{2}TF

_{2}pairings, and for each pairing we recorded the OFRs to three contrast combinations (see Methods). The fitted values of

*WR*and

*n*are shown in Figure 4A–C and 4D–F, respectively, and one can clearly see that, for a given SF pairing, the values of

*WR*and

*n*show further sizeable changes, depending on the TF of the components. Our pilot data suggested (Appendix D) that the changes in

*WR*can be well described by a simple interaction between SF and TF, where

*WR*is a Gaussian function of TF (Equation 11), but the TF at which

*WR*peaks (i.e., optimal TF) depends on SF (Equation 12):

*n*is a separable function of SF and TF, well-described with a Gaussian function of TF:

*n*for any spatiotemporal component:

*WR*, six describing

*n*; see Table 1.1). We then used the model to test five SF pairs (0.08–1.25 cpd) with the same TF (Experiment 1) and six SF/TF combinations (Experiment 2, TFs from 3⅛ to 30 Hz). OFRs to these 11 pairings were measured in three subjects. The model was then fit to each subject's data. The results of Experiments 1 and 2 are shown in Figures 5 and 6, respectively, with responses to two-component gratings shown with red circles, and the model fit shown with dashed red lines. The fits were very good:

*r*

^{2}= 0.994, 0.991, and 0.989 for subjects BMS, EJF, and TH, respectively. Figure 7 shows the fitted functions used for each subject, which are for the most part similar. The greatest intersubject variation seems to be in the effect of geometric mean SF on how rapidly

*n*falls off with SF ratio (Figure 7F).

*r*

^{2}= 0.993, 0.993, and 0.984 for subjects BMS, EJF, and TH, respectively. Table 1.2 lists the best-fit values of free parameters. Because we did not vary TF in this experiment, Equations 13 and 15 were not used. Many of the fitted parameters were very similar to those obtained in Experiments 1 and 2. Indeed, we found that we could constrain five parameters to be the same as those used for Experiments 1 and 2, with almost no change in fit quality (

*r*

^{2}= 0.994, 0.992, and 0.986 for subjects BMS, EJF, and TH, respectively), shown by grey dotted lines in Figures 5, 6, and 8. The two parameters that were substantially changed by stimulus size were

*μ*and

_{HC}*μ*: both moved to higher SF for the smaller stimulus (see Table 1.3).

_{σn}*r*values, and those fits are shown by solid (Experiment 1) and dashed (Experiment 3) grey lines. For both stimulus sizes, the peak SF of these tuning curves is very similar to the

^{2}*μ*parameter used to describe responses to two-component gratings in Equation 9. Thus, size seems to influence this parameter through its effect on single gratings. If, instead of allowing

_{σn}*μ*to be a free parameter in Equation 9, we constrain it to be the peak of the SF tuning to single gratings, we get almost no change in

_{σn}*r*s (0.993, 0.991, and 0.987 for subjects BMS, EJF, and TH, respectively). Overall then, this allows us to describe the interactions between components in 16 stimuli (32 values) with just 12 free parameters.

^{2}*μ*(fit with Equation 7) owing to changes in stimulus size is very similar to the shift of the peak of the OFR SF tuning curve. If instead of allowing

_{HC}*μ*to be a separate free parameter in each experiment we constrain it to be proportional to the peak of the SF tuning to single gratings, we get little change in

_{HC}*r*s (0.988, 0.984, and 0.987 for subjects BMS, EJF, and TH, respectively). The constant of proportionality is then just one parameter for both experiments, allowing us to describe all the data with 11 free parameters.

^{2}*W*and

_{i}*n*depend on the SF and TF of the components in the same way? When considering the second question, several of the equations (Equations 6, 7, 11, 12, and 14) can be applied without modification. Equation 9 can simply be rewritten in a way that applies to the general case:

*n*depends on the ratio of two SFs, does not readily apply to more than two components. However, a given SF ratio can also be described in terms of the variance of log

_{2}(

*SF*):

^{A}which produced OFRs of the same response amplitude as the low SF component (left column in Figure 10) or high

^{W}, which had the same weight as the low SF component (according to the model), shown in the right column of Figure 10.

*r*

^{2}= 0.965, 0.975, and 0.908 for subjects BMS, EJF, and TH, respectively. We also fit our complete model to all of the data in Experiments 1–4, and this is shown by grey dashed and dotted lines:

*r*

^{2}= 0.990 for all subjects (Figure 10). The fitted parameters are listed in Table 1.4. Clearly, the model provides an excellent account of responses to stimuli containing three components. One particularly telling case is the lower left quadrant for each subject. Here, the low and optimal frequencies moved in the same direction. The high frequency moved in the opposite direction and produced a smaller response when presented alone—it was chosen to have a high weight according to the model. Consequently, any competition that reflected the total driving signal in each direction would be dominated by the optimal + low pair. However, when the high frequency component is added (high

^{A}), it has a powerful effect—compare the purple triangle with the green triangles. Similarly, comparing all three components (green triangles) with optimal + high

^{A}(red circles) shows that adding the low frequency component (low) makes almost no difference. In this comparison, the low and high SF components here produced similar responses when presented alone, yet the effect of adding them to the other two components is very different.

*WR*describes the relative weight given to the two components, and

*n*describes how steeply the response changes with relative contrast. Large values of

*n*correspond to the winner-take-all–like outcome described in Sheliga et al. (2006), which we replicate when the components have similar SF. For widely different SFs

*n*approaches 1, which describes a linear summation.

*WR*). Importantly, we found that this WR was a separable function of SF

_{1}, SF

_{2}, implying that the weight assigned to a component is determined by its SF, not a more complex pairwise interaction. That is, the

*WR*for a pair composed of SF

_{i}and SF

_{j}is simply determined by (\(\frac{{{W_i}}}{{{W_j}}}\)). Surprisingly, we found that weight is not simply proportional to the response elicited by that component alone. Rather, as SF is increased beyond the optimal value for driving the OFR, the weight continued to increase for ∼1 octave. This finding in turn implies that the neuronal circuits determining weights must precede the stage at which motor responses to visual motion are generated.

*n*determines how steeply the OFR amplitude changes with the contrasts of the components, with values greater than 1, indicating a nonlinear interaction. This nonlinear interaction is greatest for similar SF. At first sight this might suggest that the competition is taking place within (rather than between) spatiotemporal channels (e.g., contrast gain control). However, this interaction also depends on absolute SF—dissimilar SFs interact more strongly when they are near the peak suggesting a more complex mechanism.

*n*and

*WR*:

*r*

^{2}= 0.985, 0.990, and 0.987 for subjects BMS, FAM, and JKM, respectively

^{4}(Figure 11). Equation 3 also provides good fits for all the data. Both models succeed in this case, because the individual components have similar weights.

^{5}The central SF of the moving pattern was either 0.125 (open diamonds) or 0.5 (filled circles) cpd, and it was paired with flickering samples, whose central SFs varied from 0.0625 to 4 cpd in octave increments: in effect, a SF tuning for flicker. Moving stimuli were also presented in isolation (smaller in size diamond and circle symbols), for comparison. Figure 12 shows that the flickering stimuli with central SFs in the range of 0.5 to 1.0 cpd were the most detrimental for both subjects. At the time, we were puzzled that these SFs were the most powerful, since they are higher than the optimum SF for driving the OFR. Since we have now demonstrated that the weight given to a component in Equation 1 is also biased towards higher SF, we tested whether Equation 1 offers a good description of those results:

*WR*and

*n*were calculated from Equations 13 and 15 using best fit values from Table 1.1. The predictions (no parameters were fit to the noise data) are shown by black solid (0.125 cpd data) and dotted (0.5 cpd data) lines in Figure 12. Thus, the relationship between SF and weight that we report here provides a good description of the suppressive effects attributed to flicker in our previous study (Sheliga et al., 2016).

^{2}The background color of Figure 2A-C is irrelevant and was chosen black in order to make

*WR*values of different color clearly visible.

^{3}The background color of Figure 3D-F is irrelevant and was chosen black in order to make

*n*values of different color clearly visible.

^{5}The overall stimulus size in Sheliga et al. (2016) was very similar to that in Experiments 1, 2, and 4 of the current study (25°x25° vs. 22°x22°).

*Spatial Vision*, 10 (4), 433–436. [CrossRef] [PubMed]

*Vision Research*, 15, 447–450. [CrossRef] [PubMed]

*Journal of Physiology*, 357, 219–240. [CrossRef] [PubMed]

*Current Biology*, 27 (10), 1514–1520 e1513. [CrossRef] [PubMed]

*Visual Neuroscience*, 5 (2), 107–122. [CrossRef] [PubMed]

*Visual Neuroscience*, 13 (3), 477–492. [CrossRef] [PubMed]

*WESCON Conference Proceedings*, 2 (1), 1–10.

*Program No. 460.26, 2008 Neuroscience Meeting Planner*. Washington, DC: Society for Neuroscience. Online.

*The European Journal of Neuroscience*, 25 (6), 1780–1792. [CrossRef] [PubMed]

*Neuroscience and Biobehavioral Reviews*, 36, 1–25. [CrossRef] [PubMed]

*Neuroscience Research*, 61 (1), 56–69. [CrossRef] [PubMed]

*The European Journal of Neuroscience*, 10 (3), 811–822. [CrossRef] [PubMed]

*Journal of Neurophysiology*, 56 (5), 1321–1354. [CrossRef] [PubMed]

*Dynamics of Visual Motion Processing: Neuronal, Behavioral and Computational Approaches*(pp. 141–160). New York: Springer-Verlag.

*Program No. 726.709. 2014 Neuroscience Meeting Planner*. Washington, DC: Society for Neuroscience. Online.

*Spatial Vision*, 10 (4), 437–442. [CrossRef] [PubMed]

*Journal of Neuroscience*, 37 (45), 11051–11066. [CrossRef] [PubMed]

*Institute of Electronic and Electrical Engineers: Transactions in Biomedical Engineering, BME-10,*137–145.

*Vision Research*, 45, 3307–3321. [CrossRef] [PubMed]

*Progress in Brain Research*, 171, 237–243. [CrossRef] [PubMed]

*Vision Research*, 46 (13), 2041–2060. [CrossRef] [PubMed]

*Vision Research*, 68, 1–13. [CrossRef] [PubMed]

*Journal of Vision*, 16 (1), 8. [CrossRef]

*Program No. 220.05, 2018 Neuroscience Meeting Planner*. San Diego, CA: Society for Neuroscience. Online.

*Nature Neuroscience*, 15 (11), 1596–1603. [CrossRef] [PubMed]

*Vision Research*, 43 (4), 431–443. [CrossRef] [PubMed]

*WR*(color-coded small squares; see color bar in Figure B1B) as a function of SF

_{1}and SF

_{2}in a two-component stimulus. We found that the value of

*WR*was well-described by a separable function of SF (

*r*

^{2}= 0.917). Figure B1C shows the function relating

*W*to SF

_{i}_{i}, and Figure B1B shows the resulting surface \(WR = \frac{{{W_1}}}{{{W_2}}}\), with the data squares superimposed. The function relating SF to

*W*was constructed from the product of two functions: an exponential (Equation 6; Figure B1D) and an inverted Cumulative Gaussian function (Equation 7; Figure B1E), where λ

*μ*

_{W},_{HC}

*,*and σ

_{HC}are three free parameters.

^{6}μ

_{HC}is the high cutoff frequency, where the function in Figure B1E is 0.5 (mean of the Gaussian function). σ

_{HC}describes the steepness of this cutoff.

*n*(color-coded small squares; see color bar in Figure C1B) as a function of SF

_{1}and SF

_{2}in a two-component stimulus. There are two notable features in Figure C1A. First,

*n*is largest close to the identity line (when the SF ratio is close to 1) and falls off smoothly as the SF ratio increases. We described this with a Gaussian of the

*log*SF ratio (Equation 8). Second,

*n*decreases more rapidly with the SF ratio at high and low frequency, with the slowest decline for pairs where the geometric mean SF is ∼0.25 cpd, close to the optimal SF for driving the OFR with single gratings. We were able to describe this feature successfully by allowing the value of sigma in Equation 8 to be a Gaussian function of the log of the product of SFs (Equation 9). Equation 10 provides a good fit (

*r*

^{2}= 0.909) for the relationship between SF

_{1}, SF

_{2}and

*n*, using four free parameters (see the resulting surface in Figure C1B).

^{7}

_{1}TF

_{1}, SF

_{2}TF

_{2}pairings, and for each pairing we recorded the OFRs to three to five contrast combinations (from 4 to 36%). The values of

*WR*and

*n*fit with Equation 1 are shown by symbols in Figures D1A and D1D, respectively.

*WR*as a function of TF for seven different pairs of SF (each pair shown with a different color). It is clear that the weight assigned to a component declines as the TF moves away from the optimal. Changes in the absolute value of

*WR*make it difficult to determine if this is a separable function of SF and TF. To clarify this, we exploit the fact that for 5 out of 7 SF pairings there was a condition in which TF

_{1}= TF

_{2}= 18¾ Hz. Figure D1B replots the data from Figure D1A normalized by the value of

*WR*in this condition. That is, for the case where TF

_{1}= TF

_{2}= 18¾ Hz, any differences in the

*WR*can be attributed to the values of SF alone. If the

*WR*is a separable function of SF and TF, then after this normalization a single function should describe the effect of TF. Although the dependence on TF is similar in these normalized curves, there are significant differences, especially at low TF. The error bars (68% confidence intervals) in Figure D1B are in most cases smaller than the data points. We find that these differences can be well described by a simple interaction between SF and TF (Equation 11), where

*WR*is a Gaussian function of TF, but the TF at which

*WR*peaks (i.e., optimal TF) depends on SF (Figure D1C; Equation 12). We choose an exponential in Equation 12 because it is well-behaved as the SF becomes large. The data over the range we observe could equally well be described by a straight line. We can now summarize all the factors influencing a component's weight by combining Equations 5 and 11; see Equation 13. Colored solid lines in Figure D1A show how well (

*r*

^{2}= 0.951) this model fits the pilot dataset.

^{8}Note that the fits are also very good for pairings in which TF of neither component equaled 18¾ Hz (black dotted and grey dashed lines).

*n*as a function of TF, for the same seven SF pairings as described. For the five pairs that included TF

_{1}= TF

_{2}= 18¾ Hz, Figure D1E plots

*n*normalized by the value for that condition. As with the

*WR*, the value of

*n*falls as TF deviates from the optimal. This effect is similar for all SF pairings except one (0.22/0.05 cpd), shown in green. This pair was also associated with a larger confidence interval than the other cases (because the response amplitudes were smaller), so we felt that a separable function of SF and TF would suffice to describe these data. We fit this with a Gaussian function of TF (Equation 14). The black solid line in Figure D1E shows this fit. Combined with Equation 10, this allows us to summarize the factors influencing

*n*for any spatiotemporal component: Equation 15. Colored solid lines in Figure D1D show how well (

*r*

^{2}= 0.749) this complete model fits the whole pilot dataset.

^{9}