Human visual steady-state responses to amplitude-modulated flicker: Latency measurement

Abstract

The response latency of steady-state visually evoked potentials (SSVEPs) is a sensitive measurement for investigating visual functioning of the human brain, specifically in visual development and for clinical evaluation. This latency can be measured from the slope of phase versus frequency of responses by using multiple frequencies of stimuli. In an attempt to provide an alternative measurement of this latency, this study utilized an envelope response of SSVEPs elicited by amplitude-modulated visual stimulation and then compared with the envelope of the generating signal, which was recorded simultaneously with the electroencephalography recordings. The advantage of this measurement is that it successfully estimates the response latency based on the physiological envelope in the entire waveform. Results showed the response latency at the occipital lobe (Oz channel) was approximately 104.55 ms for binocular stimulation, 97.14 ms for the dominant eye, and 104.75 ms for the nondominant eye with no significant difference between these stimulations. Importantly, the response latency at frontal channels (125.84 ms) was significantly longer than that at occipital channels (104.11 ms) during binocular stimulation. Together with strong activation of the source envelope at occipital cortex, these findings support the idea of a feedforward process, with the visual stimuli propagating originally from occipital cortex to anterior cortex. In sum, these findings offer a novel method for future studies in measuring visual response latencies and also potentially shed a new light on understanding of how long collective neural activities take to travel in the human brain.

Introduction

It has been suggested that the visual system is hierarchical in structure (Felleman & Van Essen, 1991; Hilgetag, Van Essen, Neill, Young, & Felleman, 1996). Visual input is thought to be processed along largely parallel streams, starting from the retinal ganglion cells, different layers of the dorso-lateral geniculate nucleus, and proceeding to the visual cortex (Merigan & Maunsell, 1993; Shapley & Perry, 1986). In considering this, and to understand the fundamental principles of the human visual system, it is crucial to know how long it takes for the system to process the incoming visual signal. In other words, accurate measurement of visual response latencies is desirable. 

In nonhuman primate studies, the visual response latency for different areas has been estimated based on the onset latencies of single-unit responses evoked by visual stimuli. Results have shown that lower order visual areas show shorter latencies than higher ones (Azzopardi, Fallah, Gross, & Rodman, 2003; Cowey, 1964; Nowak, Munk, Girard, & Bullier, 1995; Schmolesky et al., 1998). In human studies, onset latency has been measured by means of visual evoked potentials (VEPs) recorded using electroencephalography (EEG). The most common stimuli used to elicit VEPs involve pattern reversal, using either a checkerboard or a grating pattern (Odom et al., 2016; Regan, 1989). The resulting VEPs are characterized by positive and negative peaks, which can be measured in terms of direction, amplitude, and latency. For example, the P100 event-related potential component is a transient positive peak that occurs approximately 100 ms after the onset of the pattern reversal stimulus. In normal adults, early work showed facilitation/summation of P100 amplitude when the pattern-reversal stimuli were presented simultaneously to two eyes (binocular viewing) compared to presentation to just one eye (monocular viewing). The binocular P100 latency is also seen to be slightly shorter than the monocular P100 latency (Johansson & Jakobsson, 1993; McCulloch & Skarf, 1991; McKerral et al., 1995; Wanger & Nilsson, 1979). However, the VEPs have been found to be quite variable, affected by factors such as the color and intensity of the stimulus (Bessler, Klee, Kellner, & Haueisen, 2010; Klee, Link, Bessler, & Haueisen, 2011; Porciatti, Burr, Morrone, & Fiorentini, 1992; Regan, 1966). This variability is an obstacle to establishing relationships between VEPs and stimulus parameters. Moreover, instead of measuring time-points across the signal, researchers have typically used the peaks of VEP activity for measurements, rather than using the waveform as a whole (Lee, Birtles, Wattam-Bell, Atkinson, & Braddick, 2012; Norcia, Appelbaum, Ales, Cottereau, & Rossion, 2015). For example, the latencies have been measured from transient VEPs peaks regardless of whether the waveforms have a late start and fast rise or an early start and slow rise (Norcia et al., 2015). 

Steady-state visually evoked potentials (SSVEPs), which are seen when using a sinusoidally modulated light stimulus (Regan, 1966), can be used to make an alternative measurement of electrophysiological responses in which the amplitude response and phase response of SSVEPs can be analyzed using fast Fourier transform (FFT) of the entire waveform. This results in phase measurement of SSVEPs, which was found to be reliable between participants (Celesia, 1980; Regan, 1982). In addition, the SSVEPs obtained using this approach are less susceptible to artifacts induced by blinks and eye movements (Perlstein et al., 2003; Vialatte, Maurice, Dauwels, & Cichocki, 2010). While transient VEPs are typically used for studying the visual system, SSVEPs have been more broadly used in cognitive neuroscience, clinical neuroscience, and research looking at brain computer interfaces (for a review see Vialatte et al., 2010). Therefore, SSVEPs provide a potential means for investigating collective activity of large groups of neurons. However, the phase shift between the visual stimulus presented and the resulting SSVEP is difficult to quantify because of the steady-state nature of the response, with, for example, it being difficult to know how many preceding cycles occur in the SSVEP (for a review see Norcia et al., 2015). That is, there may be a phase shift between the stimulus and the response, which can wrap around one, two, or more times. This makes the phase measurement ambiguous for measuring the absolute latency of the response. For example, the phase shift between the stimulus and SSVEP is measured at about 25° when using FFT analysis. However, because of uncertain regarding the preceding cycles, this phase shift could be 25° or 385° (360° plus 25°). 

One approach to overcome this ambiguity is to measure the “apparent latency” introduced by Regan (1966), in which multiple-frequency stimuli are used to allow a plot of the slope of the phase versus frequency. Apparent latency has been used in early work for the evaluation of visual development or optic neuritis in clinical populations (Falsini & Porciatti, 1996; Lee et al., 2012; Morrone, Fiorentini, & Burr, 1996; Strasburger, Scheidler, & Rentschler, 1988). For example, in infant visual development, the response latency was found to be approximately 215 ms at 3.6 weeks of age and progressively shortened until approaching 100 ms (an adult-like latency) at 30 weeks of age (Lee et al., 2012). However, in order to fit the slope or “produce maximum orderliness,” a multiple of 2π radians must be added or subtracted from the phase response of higher frequencies. For instance, in Lee et al.'s study (2012), if the differential phase measurements between two adjacent frequencies was positive, the multiple integers of 2π radians were subtracted from the phase response of the higher frequency until it became negative in order to fit the slope (Lee et al., 2012). The uncertainty of adjusting an appropriate number of 2π radians may lead to an unclear relationship between “apparent latency” and the actual physiological delay (i.e., absolute latency). In addition, the slope of phase versus frequency is assumed to be a straight line that may not be suitable for the physiological visual system, which distinguishes three frequency ranges in SSVEPs (i.e., below 15 Hz, 15–25 Hz, 25–60 Hz) as a curvature of amplitude versus frequency (Bijl & Veringa, 1985; Regan, 1982). Since the physiological latency is an important indicator to investigate the temporal properties of the human visual system, an improved method thus must be considered. The apparent latency is a measure used in auditory physiology and is described as “group delay” (Goldstein, Baer, & Kiang, 1971). In engineering, group delay is the time delay between the envelope of the generator and receiver of the amplitude-modulated (AM) signal, in which the amplitude of a fast oscillation (i.e., the carrier signal) is varied by a slow oscillation (i.e., the envelope or amplitude modulation). Therefore the envelope response of SSVEPs could be used to estimate the group delay by calculating the phase shift between the visual stimuli and the response, which can consequently be considered as the physiological latency of the response in the human brain, instead of using phase slope. 

In this study, we attempted to estimate the group delay using the envelope of SSVEPs elicited by an AM flicker, a signal with a 2-Hz envelope and a 14-Hz carrier. Our hypothesis was that, using this approach, the 2-Hz envelope response of the elicited SSVEPs could be extracted and compared directly with the 2-Hz envelope of the generating photodiode signal that was simultaneously recorded with the EEG for estimation of the group delay. Similar to VEP studies, it is critical to understand the primary visual processing time from the pathways from the two eyes. Therefore, the current study estimated the group delay at the occipital channel (i.e., Oz channel) for both binocular and monocular viewing conditions. In addition, the wide distribution across EEG channels of the SSVEP allows investigation of neural populations on a whole-brain scale (Silberstein, 1995; Srinivasan, Bibi, & Nunez, 2006). We thus also estimated the group delay at frontal channels (i.e., a cluster of F3, Fz, F4 channels) to compare with the delay seen for the Oz channel. 

Materials and methods

Participants

Thirteen students with normal or corrected-to-normal vision and no history of psychiatric disorders participated in the study (six males, seven females; M = 24.23 years, SD = 3.05 years). Participants with a first-degree relative with migraine or epilepsy were excluded from this study. The participants were required to be in good condition (e.g., did not have flu or any sleeping condition) before attending the experiment and to avoid fatigue. The study was carried out in accordance with the Social and Behavioral Research Ethical Principles and Regulations of National Taiwan University and was approved by the Research Ethics Committee of National Taiwan University. Written informed consent was acquired from every participant before the experiment. 

Stimuli and procedures

The stimuli were viewed through two black tubes 13 cm in length, with one tube for each eye. Each tube contained a white light-emitting diode (LED) covered with a 4 × 4 cm diffuser plate at one end of the tube to allow presentation of a stimulus with a visual angle of ∼18.2° and a luminance of up to 39.2 cd/m², measured with a luminance meter (Konica Minolta LS-100; Konica Minolta Sensing Americas, Inc., Ramsey, NJ). The centers of the two tubes were 4.5 cm apart from each other and the device as a whole enabled presentation of different light flicker waveforms for the experiment in this study. The LEDs were connected to a 16-bit digital-to-analog converter (NI USB-6229 BNC, National Instruments, Austin, TX), allowing the LED signal to be modulated at a rate of up to 40 kHz. An integrated photodiode (BPW34, OSRAM Opto Semiconductors) was used to collect the output LED signal and this was recorded with a BioPac MP35 (Biopac Systems, Inc., Goleta, CA) to verify that the emitting signal had the desired shape. We empirically estimated the transmission time between the stimulus event (i.e., trigger onset) and the recorded photodiode signal, which showed a short and consistent delay across all the conditions (see the condition list in Table 1). Thus, the visual response latency estimated from these conditions would not be expected to be significantly influenced by the transmission time of the photodiode. 

Table 1

 

Experiment conditions.

 

Both sinusoidal (S) and AM flicker were generated using MATLAB (MathWorks, Natick, MA) in-house scripts with the following equations: 

S waveform:  

\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\begin{equation}\tag{1}S\left( t \right) = {L_0} + {L_0}\sin \left( {2\pi {f_c}T} \right)\end{equation}

 

AM waveform:  

\begin{equation}\tag{2}AM\left( t \right) = {L_0} + {L_0}\sin \left( {2\pi {f_m}T} \right)\sin \left( {2\pi {f_c}T} \right){\rm }\end{equation}

Where T was the duration of S and AM flicker, L₀ was the mean luminance, f_c was the carrier frequency, and f_m was the modulation frequency.  

For baseline comparisons, we created three control conditions (binocular condition and monocular conditions with dominant eye and nondominant eye) using a 14-Hz single frequency S flicker, with a block of 30 trials for each condition (see Equation 1 and Table 1). We also generated three testing conditions (binocular and monocular conditions) with the AM flicker using two sinusoidal signals, with one being a fast-changing oscillation (i.e., the carrier frequency, f_c = 14 Hz) and the other a slow-changing oscillation (i.e., the modulation frequency, f_m = 1 Hz; see Equation 2 and Table 1). Since AM flicker is a complex signal, we followed the recommendations of the International Society for Clinical Electrophysiology of Vision for the number of trials in SSVEP investigations (Odom et al., 2016). Accordingly, 50 trials were used for each testing condition to obtain SSVEP signals for measuring the visual response latency. Note that the envelope of this AM flicker was twice the modulation frequency (i.e., f_env = 2 Hz). Mathematically, however, this AM flicker could also be treated as the superposition of two sinusoidal waveforms (i.e., f₁ = 13 and f₂ = 15 Hz) with equal amplitudes. Overall, there was a total 240 trials from six conditions (as shown in Table 1). Trials of each of the six conditions were randomly presented. Participants were asked to press any key to initiate the first trial. After the keypress, participants were required to open their eyes when they heard a beep sound and then fixate their sight on the black point of the diffuser plate LED for 2.5 s. Presentation in 2.5-s trials was adopted to avoid signal variations due to fatigue or discomfort resulting from prolonged inspection of flicker. Afterward, participants could take a rest for 2 s (Figure 1). Another beep occurred to indicate the start of the next trial. 

Figure 1

 

Schematic diagram and procedure of the SSVEP experiment. (A) Schematic diagram of the SSVEP recording system. The computer, using MATLAB, was used for generating the light flicker stimuli and collecting the SSVEP and photodiode signals. To generate the visual stimuli, the computer sent the waveform output to a digital-to-analogue converter (NI USB 6229), which was connected to the two LEDs. The SSVEP and photodiode signals were recorded using a NuAmp amplifier and a BioPAC amplifier, respectively. To synchronize the time onset from these two systems, the computer simultaneously sent digital triggers (i.e., TTL (Transitor Transitor logic) triggers) to each system. (B) The experimental procedure for monocular and binocular stimulation in the experiment. After hearing a beep sound, subjects were required to open their eyes. The visual stimuli were presented to participants at random with a time duration of 2500 ms. After stimulation, participants could close their eyes until the beginning of the next trial, the start of which was again indicated by a beep sound. The visual stimuli were presented to both eyes (binocular), only to the left eye, or only to the right eye (monocular) across conditions.

EEG data acquisition and preprocessing

An elastic cap (Electrocap International) containing 36 Ag/AgCl electrodes arranged according to the International 10-20 system was used to obtain the EEG activity that was recorded using a Neuroscan amplifier (NuAmps Digital Amplifier, Model 7181) and Neuroscan 4.2 software with a sample rate of 1000 Hz. All the data was referenced to the right and left mastoids. The impedance for every electrode was kept below five kΩ during the recordings. All data were first filtered with a bandpass filter of 0.5–50 Hz (using default settings, Butterworth 5th, zero-phase filter, in the SPM8 toolbox). The data were then epoched from 0 to 3000 ms relative to stimulus onset for each trial and then detrended (i.e., the mean and linear trend were removed) before excluding trials with blinks or horizontal eye movements (exceeding 100 uV, determined by visual inspection and also followed the suggestions from Luck, 2014). Afterward, the preprocessed data were averaged across trials to obtain the SSVEPs. Finally, the SSVEP responses from the Oz channel were used as the main source of the evoked response for further data analysis (Bianciardi et al., 2009; Di Russo et al., 2007; Vialatte et al., 2010). Specifically, we used the SSVEP windows from 500 to 2500 ms after the onset of each stimulus to exclude the VEP and to increase the signal to noise ratio of the SSVEP (Andersen, Hillyard, & Muller, 2013; Andersen & Müller, 2015). SPM8 (Wellcome Trust Centre for Neuroimaging; https://www.fil.ion.ucl.ac.uk/spm/), FieldTrip (Oostenveld, Fries, Maris, & Schoffelen, 2011; http://www.ru.nl/donders/fieldtrip/), and customized MATLAB codes (MathWorks) were utilized for further data analysis at a sensor level. Standardized low resolution electromagnetic tomography was used for source level analysis. 

Latency estimation based on the envelope response

The essence of this method was to extract the envelope of the photodiode and SSVEP response using complex demodulation (Draganova & Popivanov, 1999; Kashiwase, Matsumiya, Kuriki, & Shioiri, 2012) and then calculate the phase lag between the envelope of SSVEP and photodiode signal. 

Four steps were implemented to calculate the time delay (as illustrated in Figure 2): 

1.  
   Complex demodulation was used to extract the envelope of SSVEPs in AM and S flicker conditions. The SSVEP data were multiplied (i.e., pointwise multiplication) by the complex exponential function e^–i2πft to move the frequency of interest (f = 14 Hz) to zero in the frequency spectrum. This step is different from the Fourier transform in which a dot product is used. The result of a dot product is a scalar. In contrast, here the result of multiplication (i.e., pointwise multiplication) is a vector. Next, a low-pass filter was used to extract the envelope of SSVEP responses. To get a smooth amplitude demodulation (i.e., envelope) for phase calculations, we used a low-pass filter (cutoff frequency at 3 Hz, Butterworth 3rd zero-phase filter) to keep the fundamental amplitude modulation (i.e., f_m = 1 Hz) and removed high oscillation frequencies.
2.  
   The envelopes were band-pass filtered at 1–3 Hz centered at 2 Hz using a 3rd order Butterworth zero-phase filter to acquire a zero-crossing filtered envelope. Next, a Hilbert transform was applied to get the instantaneous phases from the filtered envelope of the photodiode signal (φ_{env_photo}) and SSVEP response (φ_{env_ssvep}). The instantaneous amplitudes were only calculated for the filtered envelope of SSVEP response and summed to plot the amplitude spectrum in the frequency domain.
3.  
   The instantaneous time delays over 2,000 time points were calculated in milliseconds according to the formula:
    
    

   \begin{equation}\tag{3}{T_{{delay}}} = {{1000 \times ({\varphi _{{env\_photo}}} - {\varphi _{{env\_ssvep}}})} \over {2 \times \pi \times {f_{{env}}}}}\end{equation}

   Where φ_{env_ssvep} , φ_{env_photo} represent the unwrapped instantaneous phase of envelope SSVEP responses and envelope photodiode recordings respectively. f_env is the frequency of the filtered envelope corresponding to 2 Hz.
4.  
   The instantaneous time delays were averaged to quantify the average time delay (group delay).

Figure 2

 

The algorithm flow used to calculate the visual response latency from the envelope response. The envelopes of the signals are extracted using complex demodulation, which is illustrated in the inset. In this, the signal S was a product of two sinusoidal signals, with one being a fast oscillation at 14 Hz and one signal a slow oscillation at 1 Hz. Mathematically, this signal S was treated as a summation of two sinusoidal signals at 13 and 15 Hz with equal amplitudes. The signal S was then multiplied by the complex exponential function e^–i2πft. The real part and imaginary part of the resulting signal were filtered with a low-pass filter (cutoff frequency at 3 Hz, Butterworth 3rd zero-phase filter). The filtered real part and imaginary part were then integrated to establish the envelope. The detailed procedure of the complex demodulation is described in more detail in Kashiwase et al. (2012). A 1–3-Hz band-pass filter was then applied to these envelopes before calculating the instantaneous phase using Hilbert transform. The phase lag between the envelope SSVEP and envelope photodiode was then converted to group delay.

Envelope-based phase lag calculated from simulated signals

To ensure that our envelop-based phase lag method can evaluate the effectiveness in calculating a latency value, we evaluated the various time delays using constructed simulated signals (Figure 3). The stimulus simulation (S, black line) was generated by a product of two sinusoidal signals, in which the fast-changing oscillation was the carrier frequency (i.e., f₂ = 14 Hz) and the slow-changing oscillation was the modulation frequency (i.e., f₁ = 1 Hz). Note that the envelope of this AM flicker is twice the modulation frequency (i.e., f_env = 2*f₁ = 2 Hz). The response simulation (R, red line) was constructed from a 100-ms delay of signal S. The envelopes of these signals were extracted using the complex demodulation method. Then, the 3rd Butterworth (1–3 Hz, BPF) was used to demean and centered the signals at 2 Hz before applying the Hilbert transform to get instantaneous amplitude and phase. The distribution of phase lags was calculated by subtracting the unwrapped instantaneous phase between S-R. The phase lag was then used to convert to time delay, in which the mean time delay between S-R were 99.6 ms. This estimated time delay was close to the ideal time delay (100 ms). We further tested how the latency estimation was affected under different noise levels, which were controlled by signal-to-noise ratio (SNR). The same simulation, as shown in Figure 3, was repeated, with adding different SNR values into the response simulation signal. The SNR values were decreased from 8 dB to −2 dB, in which the negative value means that the noise power is larger than the signal power. Our results demonstrate that although the mean estimated latency in different SNR values was slightly decreased, these latencies retained values close to the ideal latency seen in the results from a noiseless signal (Table 2). 

Figure 3

 

Illustration of measuring group delay in the simulated signals. The signal S (black line) was an AM signal generated by a product of two sinusoidal signals in which one sinusoidal signal was a fast oscillation (f₂ = 14 Hz) and one sinusoidal was a slow oscillation (f₁ = 1 Hz). The signal R (red line) was the same as the signal S with a 100-ms delay. The envelopes of simulated signals were extracted by the complex demodulation. The instantaneous phase of these filtered envelopes (1–3 Hz, 3rd Butterworth zero-phase filter) were obtained by applying the Hilbert transform. The phase lag distributions between envelopes of S-R were computed in 30 bins from the 2,000 time points of instantaneous phase lag.

Table 2

 

The latency estimation with different noise levels.

 

The envelope-based phase lag from a single participant

An example of calculating the group delay from a single participant's SSVEPs elicited by AM flicker during binocular stimulation at Oz channel is shown in Figure 4. The envelope response of the SSVEP showed clear original peaks and troughs corresponding to the envelope stimulus. From the topographic distribution, the 2-Hz envelope amplitude was strongly distributed in occipital channels (O1, O2, Oz channels) while the visual response latency was 86.07 ms. 

Figure 4

 

Example of latency and spectrum data from a single participant. (A) The photodiode signal (black line) and its envelope (green line) are shown from the onset and last for 3000 ms. In addition, the SSVEP response (blue line) and its envelope (red line) were also plotted. The 1–3-Hz envelope response of photodiode and SSVEP showed the time lag over time (bottom figure). The 1–3-Hz envelope amplitude also showed the 2-Hz peak in the frequency spectrum (marginal Hilbert transform). (B) The instantaneous phase of these envelopes and the phase lag distribution showed the different phase lag between stimulus and response. This phase lag was then converted to a time delay with 86.07 ms.

Statistical analysis

In general, experimental manipulations were within-subject factors. The results were expressed as mean ± standard error of the mean. The Shapiro-Wilk test was used to test the normality of the distributions and indicated that the data did not significantly differ from a normal distribution. Differences between experimental conditions were thus assessed using paired-samples t tests, or via repeated-measures analyses of variance (rmANOVA). A Bonferroni correction was performed for the adjustment of multiple comparisons. Cluster-based nonparametric permutation (CBnPP; Maris & Oostenveld, 2007) analysis was used for evaluation of the multichannel amplitude differences from two conditions (AM flicker and S flicker). Two EEG sensors were defined as neighbors if their distance was no more than 60 mm from each other. We then proceeded with 2,000 permutations for each test. 

Source localization analysis

Prior to source reconstruction, the SSVEPs for each condition were re-referenced to a common average reference. Then the current source density of the SSVEPs at the sensor level was computed using the LORETA-Key software (KEY Institute for Brain–Mind Research, Switzerland). In this software, standardized low-resolution electromagnetic tomography (sLoreta) was applied to obtain the current source density in 6239 voxels, with a 5 mm³ resolution (Pascual-Marqui, 2002). The sLoreta used a 3-D head model, registered to the anatomical Talairach atlas (Talairach & Tournoux, 1988) and available as the digitized probability atlas from the Brain Imaging Center at the Montreal Neurological Institute (Collins, Neelin, Peters, & Evans, 1994). To extract the source envelope from the current source density, we applied the same procedure at the sensor level. Current source density of a 2-Hz envelope amplitude in each voxel between the AM flicker (test condition) and S flicker (control condition) were compared by permutation tests on paired data. For this comparison, sLoreta used a nonparametric permutation test with 5,000 randomizations (Nichols & Holmes, 2002). The threshold was set to p < 0.05. 

Results

The propagating time (group delay) from the stimulus presentation to the Oz channel during binocular and monocular stimulation

The topographic distribution of 2-Hz envelope amplitude elicited by AM flicker and S flicker are shown in Figure 5. In the AM condition, the 2-Hz envelope amplitude was strongly distributed in the occipital area in both monocular and binocular conditions. In contrast, the topographic distribution of the 2-Hz envelope amplitude in the S flicker condition (i.e., baseline condition) was not seen. The topographic contrast showed a significant difference in envelope amplitude between the AM and S flicker conditions at the occipital lobe in the monocular and binocular conditions (p < 0.05, two-tailed cluster-based nonparametric permutation). 

Figure 5

 

The scalp topographies (32 channels) of the amplitude of the 2-Hz envelope during binocular (left panel) and monocular stimulation (dominant eye: middle and nondominant eye: right panel), averaged across participants (N = 13). The topographical contrast with white circles illustrates where there were significant differences of envelope amplitude between the AM flicker condition and the S flicker condition (p_cluster < 0.05, two tails).

The Oz channel was thus chosen to calculate the visual response latency for both binocular and monocular stimulations using the envelope-based latency method (Figure 6). From visual inspection, the grand average SSVEPs induced by AM flicker showed the same amplitude modulation pattern in both monocular and binocular stimulation. These patterns in both monocular and binocular stimulation showed the time delay compared to the recording photodiode (Figure 6A). However, the larger amplitude can be seen in the SSVEPs resulting from binocular stimulation. The envelope of SSVEPs were then extracted by the complex demodulation method and also showed a time lag compared to the envelope of the photodiode signal (Figure 6B). Figure 6C shows the 2-Hz peaks of the marginal amplitude spectrum of the SSVEP envelopes. The calculated amplitudes were 5.3 ± 0.67 uV (M ± SEM) for the dominant eye, 4.99 ± 0.64 uV for the nondominant eye, and 10.58 ± 0.82 uV for binocular stimulation. One-way rmANOVA was used for the 2-Hz envelope amplitude in three conditions, showing that the difference was significant, F(2,24) = 69.98, p < 0.001, η² = 0.854). Furthermore, post hoc pairwise comparisons showed that the 2-Hz envelope amplitude during binocular stimulation was significantly larger than for dominant eye, t(12) = 7.77, p < 0.001, and nondominant eye, t(12) = 11.67, p < 0.001. However, there was no significant difference in 2-Hz envelope amplitudes between dominant and nondominant eye stimulation, t(12) = 0.043, p = 0.967. The calculated time lags were 97.14 ± 4.25 ms (M ± SEM) for the dominant eye, 104.75 ± 2.64 ms for the nondominant eye, and 104.55 ± 3.12 ms for binocular stimulation. One-way rmANOVA showed no significant difference between these, F(2,24) = 2.61, p = 0.094, η² = 0.179 (Figure 6D). 

Figure 6

 

The envelope amplitude and visual response latencies calculated using the phase-based method for monocular and binocular stimulation. (A) The photodiode signal (black line) and grand average of SSVEP responses elicited by stimulation of the dominant eye (green line), nondominant eye (blue line), and binocular stimulation (red line) are shown from 500 ms after onset and last for 2000 ms. (B) The envelope response of the photodiode and SSVEPs. (C) The spectrum shows the 2-Hz peak envelope amplitude (marginal amplitude from Hilbert transform) from monocular and binocular stimulation. The binocular response (red line) shows a significantly larger amplitude compared to dominant (green line) and nondominant eye (blue line) monocular stimulation. (D) The visual response latency for the dominant eye (97.14 ± 4.25 ms), nondominant eye (104.75 ± 2.64 ms), and binocular stimulation (104.55 ± 3.12 ms). Error bars represent standard error of the mean.

The propagating time (group delay) for the frontal lobe channels was longer than for the occipital channels during binocular stimulation

To calculate the response latency at the frontal electrode cluster (F3, Fz, F3 channels), the envelope amplitudes at this area elicited by AM flicker was assumed to be significantly larger than those elicited by S flicker (control condition). However, these envelope amplitudes at frontal channels are only significantly larger than those of S flicker in binocular stimulation and not in the monocular condition (Figure 5). Therefore, the response latency at frontal channels was only calculated for the binocular condition. 

In the binocular condition, the visual response latencies of SSVEP responses were calculated from three regions (Occipital electrode cluster—O1, Oz, O2; central electrode cluster—C3, Cz, C4; frontal electrode cluster—F3, Fz, F4), with the SSVEPs of the three channels from each region being averaged (Figure 7). The grand average SSVEP responses induced by binocular AM flicker for the three regions are shown in Figure 7A. In addition, visual inspection of these SSVEPs showed a time delay compared to the recording photodiode. The envelope of SSVEP responses in the three region channels (blue line: occipital channels, green line: central channels, red line: frontal channels) are shown in Figure 7B. The envelope responses at frontal channels shows a lower amplitude and a longer delay than in occipital channels. The 2-Hz envelope amplitudes were calculated as 9.27 ± 0.78 μV (M ± SEM) for the occipital channels, 2.02 ± 0.3 μV for the central channels and 2.62 ± 0.31 μV for the frontal channels. One-way rmANOVA showed that the difference in 2-Hz envelope amplitude in the three region channels was significant, F(1.069, 12.833) = 88.528, p < 0.001, η² = 0.881 (Figure 7C). Post hoc pairwise comparisons with Bonferroni correction showed that the 2-Hz envelope response at occipital channels was significantly larger than those of central, t(12) = 9.349, p < 0.001, and frontal channels, t(12) = 9.787, p < 0.001. Post hoc tests also showed significantly lower amplitude for central channels compared with frontal channels, t(12) = −3.229, p = 0.022. The time lags were calculated as 104.11 ± 4.25 ms (M ± SEM) for the occipital channels, 124.29 ± 8.15 ms for the central channels and 125.84 ± 6.89 ms for the frontal channels. One-way rmANOVA revealed a significant main effect of channel on latency responses, F(1.3, 15.7) = 6.417, p = 0.016, η² = 0.348 (Figure 7D). Post hoc pairwise comparisons with Bonferroni correction showed that the latency at occipital channels was significantly shorter than that of frontal channels, t(12) = −3.72, p = 0.009. 

Figure 7

 

The envelope amplitude and visual response latency obtained using the envelope-based delay method for three region channels (occipital, central, and frontal channels). (A) The photodiode signal (black line) and grand average of the SSVEP response elicited in occipital channels (blue line), central channels (green line), and frontal channels (red line) are shown from 500 ms after onset and last for 2000 ms. (B) The envelope response of the photodiode and SSVEPs. (C) The frequency spectrum showing the 2-Hz peak envelope amplitude (marginal Hilbert transform) in the three region channels. The occipital channels (blue line) shows a significantly larger amplitude compared to central (green line) and frontal channels (blue line). (D) The visual response latency in occipital channels (104.11 ± 4.25 ms), central channels (124.29 ± 8.15 ms), and frontal channels (125.84 ± 6.89 ms). Error bars represent standard error of the mean.

Using sLORETA, we estimated the current source density for the 2-Hz source envelope in the AM condition and the S flicker condition (control condition). In the AM condition, the source localization demonstrated that there was a significant increase in the current source spectral density in the occipital cortex (p < 0.05 at critical t value > 20.6; Figure 8). 

Figure 8

 

Source analysis of envelope amplitude in response to binocular stimulation using sLORETA. sLORETA images for the 2-Hz envelope elicited by AM flicker (top panel), showing the strong activity in the occipital lobe, and S flicker (middle panel). sLORETA statistical nonparametric maps (bottom panel) comparing the current source spectral density (J²) for the 2-Hz envelope elicited by AM flicker and S flicker. A significant increase was seen in the occipital lobe (critical t value > 20.6, p < 0.05). Calibration bars indicate t values.

Discussion

In the current study, we quantified the latency from the envelope of SSVEPs induced by AM flicker. The results indicated that the latency at the occipital channel was not significantly different between binocular and monocular conditions. In contrast, the envelope amplitude of SSVEPs in the binocular condition was larger than that in the monocular condition. These results suggest that the enhanced amplitude in binocular viewing did not shorten the latency in the binocular condition. We then estimated the latency from frontal channels in both binocular and monocular conditions. However, the envelope amplitude observed at frontal channels was large enough for the binocular condition but was not for the monocular condition compared to the baseline (i.e., envelope amplitude of S flicker). Therefore, only the binocular condition was considered for calculating the latency at frontal channels. The results showed that the latency at the frontal channels was significantly longer than that at occipital channels. These findings mean the envelope of SSVEPs may be effective for estimating the propagation time of visual stimuli to many cortical areas at a sensory level. 

Some previous studies used primarily the complex demodulation method as an important tool to extract the envelope of SSVEPs (Kashiwase et al., 2012; Müller, Andersen, & Keil, 2008; Müller et al., 2006). The envelope was then used to estimate the temporal dynamics of attention modulation. The consistent results across these studies show that this method was a reliable tool to extract the envelope of a physiological signal, specifically SSVEPs. Consistent with these studies, the current study successfully obtained the envelope of SSVEPs along with the envelope of the physical stimulus employed. Unlike the previous studies, which obtained the envelope with unknown frequency from a single frequency stimulus, the current study extended the use of envelope SSVEPs by applying an AM flicker in which the frequency of the envelope was well defined. This envelope frequency is an important parameter that allowed us to convert the phase lag obtained using Hilbert transform into the time delay. The Hilbert transform is useful for analyzing nonstationary signals (i.e., EEG data) by calculating a rate of change in phase, which is the instantaneous phase. Although the Fourier transform might offer a simpler way to calculate this phase lag, it has constraints in obtaining the instantaneous phase to precisely measure the visual response latency. The instantaneous values allow a meaningful representation of the phase and amplitude in the temporal domain, hence providing an accurate representation of nonstationary signals not observed in the integral transforms of Fourier transform. The efficiency of instantaneous phase in measuring the phase lag index has been demonstrated by Stam, Nolte, and Daffertshofer (2007). The combination of complex demodulation and Hilbert transform was validated in different noise conditions using simulations for measuring the phase lag. The results from simulations showed that the estimated latencies at different noise levels were close to the ideal latency, suggesting the validity and high performance of this method. Therefore, the combination of complex demodulation and Hilbert transform in the present study can be reliably applied to quantify the propagating time of envelopes from the physical stimulus generator to the various cortical areas at the sensory level of the human brain. 

In the literature, the successful use of the VEP for the measurement of the latency of cortical responses is possible due to the feasibility of the transient peak latency of the P100 component, which is typically approximately 100 ms (for a review see Odom et al., 2016). This P100 component peak latency is correlated with the phase change of SSVEPs (Tobimatsu, Kurita-Tashima, Nakayama-Hiromatsu, & Kato, 1993, Tobimatsu, Tashima-Kurita, Nakayama-Hiromatsu, & Kato, 1991). This relationship was supported by one study measuring apparent latency (Lee et al., 2012). In this study, 12 different frequency stimuli were used to estimate the apparent latency using linear regression of phase versus frequencies. They found that, in normal adults, the apparent latency was about 103.6 ± 3.0 ms and was not significantly different from the transient peak latency (104.6 ± 1.7 ms). In general, this finding suggested that the apparent latency was approximately 100 to 115 ms and similar to the P100 peak time in VEP studies (Lee et al., 2012). In agreement with the time range of these studies, our results found that the propagating time (group delay) from the visual stimuli generator to the occipital channel was 104.55 ± 3.12 ms (M ± SEM) for binocular stimulation, 97.14 ± 4.25 ms for the dominant eye, and 104.75 ± 2.64 ms for the nondominant eye in monocular stimulation. Although the above group delays were in consistent with the apparent latency, the envelope-based delay method has some advantages. Firstly, this method successfully estimated the latency based on the physiological envelope in the entire waveform which might be close to the physiological latency. The apparent latency method uses the slope of phase versus frequencies, in which the prior phase cycles of higher frequency have to be arbitrarily added in order to fit this slope. This arbitrary selection of phase response may lead to blurr the estimation of the physiological latency. Additionally, the apparent latency method requires runs of multiple-frequency stimuli in many blocks to allow plotting of this regression line (Falsini & Porciatti, 1996; Lee et al., 2012; Morrone et al., 1996; Porciatti & Sartucci, 1996; Regan, 1966). This can be quite time consuming. In the current study, the latency based on the envelope phase needs only one block of EEG recording elicited by AM flicker and so is experimentally significantly less time-consuming. AM stimuli have also been used in brain-computer interfaces, in which the phase and amplitude from this stimulus can be used to improve the performance of classification (Chang, Baek, Lee, & Park, 2014; Lopez-Gordo, Prieto, Pelayo, & Morillas, 2010). Therefore, the envelope-based delay method using the AM flicker is very suitable for empirical application in, for example, clinical studies and brain-computer interfaces. Moreover, homogenous flicker stimulation, rather than pattern, was adopted in this study to avoid problems that might arise from the nonsinusoidal nature of usual pattern-onset or -reversal stimuli. However, pattern stimuli give more insight into visual processing beyond the early stages of the visual pathway, and we are currently planning the application of our technique to SSVEPs elicited by such stimuli. 

To our knowledge, the binocular latency and monocular latency from SSVEP studies have previously rarely been discussed. In contrast, the P100 peak time of binocular and monocular VEPs have been intensively discussed, and the results typically show shorter binocular latencies than monocular latencies (Johansson & Jakobsson, 1993; McCulloch & Skarf, 1991; McKerral et al., 1995; Wanger & Nilsson, 1978). In contrast with these studies, the current observations showed no significant difference in binocular and monocular latency. One plausible reason for this discrepancy was the different measurement of latency. That is, the VEPs only measured the latency onset at the peaks, while the current study measured the latency by use of an entire waveform. In addition, the stimulus conditions might account for this discrepancy. For example, Bagolini, Porciatti, and Falsini's study from 1988 used a range of spatial frequencies of checkerboard stimuli and found that the shorter latency for binocular viewing compared to monocular viewing only occurred at 0.6–2 cycles per degree but no phase was shorter at spatial frequencies higher than 2 c/d (Bagolini et al., 1988). In addition to latency information, we found that the binocular envelope amplitude was larger than the envelope amplitude from stimulation of the dominant eye or the nondominant eye (see Figure 6C). This finding was generally in line with most previous studies, in which the performance of both eyes is enhanced compared to one eye in both psychophysical and electrophysiological studies (Apkarian, Nakayama, & Tyler, 1981; Baker, Lygo, Meese, & Georgeson, 2018; Baker, Wallis, Georgeson, & Meese, 2012; Campbell & Green, 1965; Di Summa et al., 1997; Ding & Levi, 2017; Ding & Sperling, 2006; Meese, Georgeson, & Baker, 2006; Plainis, Petratou, Giannakopoulou, Atchison, & Tsilimbaris, 2011; Richard, Chadnova, & Baker, 2018; Tobimatsu & Kato, 1996). The binocular summation ratio is widely used in most previous studies when measuring the binocular advantage over monocular sensitivities. In early psychophysical work, Campbell and Green (1965) reported that the binocular contrast sensitivity was about 1.4 times enhancement that of monocular sensitivity, indicating a binocular summation ratio of 1.4 (Campbell & Green, 1965). However, this ratio has been reported to be significantly greater than the canonical value of 1.4, with an approximate range from 1.4 to 2 (Baker et al., 2018; Baker et al., 2012; Meese et al., 2006; Richard et al., 2018). From the pattern of our results, the amplitudes under binocular condition were approximately 2.2 ± 0.16 (M ± SEM) times (i.e., a ratio of binocular amplitude to the averaged monocular amplitudes, averaged across participants) larger than those obtained in the monocular condition. This result supports the range from recent theories instead of the canonical ratio (i.e., a binocular summation ratio of 1.4). This was also consistent with the range from 1.3 to 2.5 reported in most previous electrophysiological studies (Apkarian et al., 1981; Di Summa et al., 1997; Plainis et al., 2011; Tobimatsu & Kato, 1996). The current finding implies an approximately linear summation process occurs during binocular viewing. 

Another important observation in the present study was that the envelope response at frontal channels in the binocular condition became smaller and was significantly delayed compared to the responses in occipital channels (Figure 7). It is noted that the latency at frontal channels during monocular stimulation was not calculated because the envelope amplitudes were not different compared to the baseline (S flicker condition). The envelope amplitude being enhanced or converging from two monocular pathways could be one plausible reason for the stronger envelope amplitude at frontal channels in binocular viewing. At the occipital channels, the envelope amplitude was 9.27 ± 0.78 μV and the latency was 104.11 ± 4.25 ms. In contrast, at the frontal channels, the envelope amplitude was 2.62 ± 0.31 μV and the latency was 125.84 ± 6.89 ms. These findings were combined with the source envelope being activated more strongly in occipital cortex and lends support to bottom-up theories that SSVEP signals in visual cortex propagate forward to frontal areas at successively later times. Although this occipital source, which was the result of the amplitude spectrum, is difficult to use to make a conclusion about the relationship between the latency difference and the source findings, we speculate that if the source is strongly activated at occipital lobe, it might imply that the envelope responses probably originated from the occipital area and then propagated to the frontal lobe at a delayed time, with this delay being the latency difference we observed between occipital and frontal channels. However, our source analysis showed no direct evidence of a frontal source for the response to the envelope. Indeed, these observations were in line with previous studies showing that the occipital lobe was the main source of visual responses (Bianciardi et al., 2009; Di Russo et al., 2007; Vialatte et al., 2010). In addition, the previous studies suggested that SSVEPs also were a means of observing traveling waves from the occipital channels to anterior channels (Burkitt, Silberstein, Cadusch, & Wood, 2000; Srinivasan et al., 2006). For example, Burkitt et al. (2000) found that a negative phase gradient of SSVEPs in most subjects was consistent with propagation in the posterior to the anterior direction (Burkitt et al., 2000). The current study also supports early evidence of traveling waves in single-unit studies. The earliest such work showed that the largest potentials were measured at the V1 surface locations that were retinotopically appropriate while smaller potentials were measured at more distal locations (Cowey, 1964). In addition, stimuli placed further away caused potentials that were progressively delayed. This early finding was confirmed by studies that measured local field potentials (Kitano, Niiyama, Kasamatsu, Sutter, & Norcia, 1994) and used voltage-sensitive dye imaging (Grinvald, Lieke, Frostig, & Hildesheim, 1994; Muller, Reynaud, Chavane, & Destexhe, 2014; Sharon, Jancke, Chavane, Na'aman, & Grinvald, 2007). 

In conclusion, the envelope-based phase delay method described here extracts the envelope response and envelope stimuli to estimate the latency in the entire waveform resulting from presentation of a fluctuating stimulus. Therefore, this method could overcome the phase ambiguity of the fast oscillation of SSVEPs and be innovatively applied to quantify visual response latencies in the human visual system. In addition, this method also provides a reliable means to observe the progressive delay from the occipital channels to frontal channels. Our current study suggests that SSVEPs and their envelope waveforms could be used to enhance current theories relating to visual response latencies and might also offer physiological evidence for traveling waves in the human brain. 

Acknowledgments

This work was sponsored by the Ministry of Science and Technology, Taiwan (grant numbers: 108-2639-H-008-001-ASP; 108-2321-B-075 -004 -MY2, 107-2410-H-008 -040 -MY3; 106-2628-H-008-002-MY4; 106-2410-H-008-038-MY3; MOST 107-2628-H-008-002-MY3; MOST 107-2420-H-008-009), Academia Sinica, Taiwan (AS-108-TP-C02-2) and sponsored by Taiwan Ministry of Education's “Academic Strategic Alliance: Taiwan and Oxford University” project grant (MOE Oxford- NCU-BRC collaborative project). We are grateful to Kia Nobre, Mark Woolrich, and Andrew Quinn for the insightful discussion. KTN, CHJ, NGM, and WKL designed the experiments. KTN collected the data. KTN, CHJ, WKL, and NEH analyzed data. All authors wrote the manuscript. The data that support the findings of this study will be available from the corresponding author upon request. 

Commercial relationships: none. 

Corresponding author: Chi-Hung Juan. 

Email: . 

Address: Institute of Cognitive Neuroscience, National Central University, Jhongli City, Taiwan.