Human vision begins when light from a retinal image is absorbed by photopigments present in the outer segments of rod and cone photoreceptors in the retina (
Grünert & Martin, 2020;
Jindrová, 1998;
Sung & Chuang, 2010). Quantal absorption within the photoreceptors gives rise to electrochemical signals whose integration in functionally and anatomically specialized pathways provides visual cortex with information on brightness, spatial frequency, color, and other dimensions of visual experience (
Hubel & Wiesel, 1962;
Derrington et al., 1984;
Krauskopf et al., 1982;
Webster & Mollon, 1994). At scotopic light levels (e.g., starlight), vision is mediated primarily by rods. Although absent from the fovea, rods are most numerous of the photoreceptors and otherwise widely distributed across the retina. At photopic light levels (e.g., daylight), the rod photopigment becomes saturated, and it is the short (S), medium (M), and long (L) wavelength light-sensitive cones that mediate vision. Cone cells are packed densely into the fovea and distributed more sparsely elsewhere. At mesopic light levels (e.g., full moon, nighttime urban lighting), vision is served by complex interactions of signals from rod and cone photoreceptors (
Zele & Cao, 2015).
Humans also possess a third class of photoreceptor. A relatively small population of intrinsically photosensitive retinal ganglion cells (ipRGCs) express the photopigment melanopsin in their axons and soma (
Provencio et al., 2000). The ipRGCs provide an additional light-sensing pathway with an important role in “nonvisual” functions, including circadian photoentrainment and control of pupil size, via their direct projections to the suprachiasmatic nucleus of the hypothalamus and the olivary pretectal nucleus of the midbrain, respectively (
Gamlin et al., 2007;
Ruby et al., 2002). Although ipRGCs do not mediate vision in the same way as rods and cones, laboratory studies have shown that melanopsin activation can facilitate the processing of cone signals and influence brightness perception (
Allen et al., 2014;
Brown et al., 2012;
Davis et al., 2015).
As illustrated in
Figure 1, the photoreceptor classes have different spectral sensitivities. The curve for each type of photoreceptor describes the probability of it capturing a photon at a given wavelength, so S cones are about 10 times more likely than L cones to capture photons at 450 nm, and the likelihood of L and M cones capturing at 550 nm is roughly the same. Because the spectral sensitivities of the photoreceptors overlap, most lights in the visible spectrum will stimulate all types of photoreceptors, albeit to varying degrees. However, with the method of silent substitution, it is possible to prepare light stimuli that selectively target individual photoreceptor classes (
Estévez & Spekreijse, 1982).
Silent substitution is an elegant technique that involves using a spectrally calibrated multiprimary stimulation system to produce spectra that selectively stimulate one class of retinal photoreceptor while maintaining a constant level of activation in the others. The method exploits
Rushton's (1972) principle of univariance, which states that the output of a photoreceptor is one-dimensional and depends on quantum catch, not on which quanta are caught. In other words, different spectra will have an identical effect on a photoreceptor, providing they lead to the same number of photons being absorbed. The principle of univariance and its relevance to silent substitution is covered in greater detail by
Estévez and Spekreijse (1982), along with a treatment of the method's history. In vision science, silent substitution has contributed to our understanding of human color vision mechanisms (
Horiguchi et al., 2013), and it has enabled researchers to examine how targeted photoreceptor stimulation affects physiological responses such as the electroretinogram (
Maguire et al., 2017) and the pupillary light reflex (
Spitschan et al., 2014).
The method of silent substitution can be used to generate metamers: spectra that have different wavelength distributions but produce the same cone excitation (the principle underlying all modern color display devices). Such lights can be used to stimulate pathways contributing to “nonvisual” responses to light, including melatonin suppression (
Allen et al., 2018;
Blume et al., 2022;
Souman et al., 2018), sleep (
Blume et al., 2022;
Schöllhorn et al., 2023), and other neuroendocrine and circadian functions (
Zandi et al., 2021). The method of silent substitution can also be used to investigate the contribution of melanopsin signaling to canonical visual processing (
Allen et al., 2019;
Brown et al., 2012;
DeLawyer et al., 2020;
Spitschan et al., 2017;
Uprety et al., 2022;
Vincent et al., 2021), and its potential as a diagnostic tool for retinal disease has garnered attention in recent years (
Kuze et al., 2017;
Wise et al., 2021).
The main hardware requirement for silent substitution is a spectrally calibrated stimulation system with at least as many primaries as there are photoreceptors of interest. With three cone classes in addition to rods and ipRGCs, this generally means that five primaries are needed, but four may suffice when working in the photopic range and sufficient measures are taken to ensure the rods are saturated (see
Adelson, 1982;
Aguilar & Stiles, 1954;
Kremers et al., 2009;
Shapiro, 2002;
Sharpe et al., 1989). The primaries should be independently addressable, additive, and ideally stable over time with a linear input–output function. Peak wavelength and bandwidth of the primaries are key considerations that will define the gamut and available contrast (
Evéquoz et al., 2021), and the light source should be integrated into an optical system for stimulus delivery (
Barrionuevo et al., 2022)—usually a Ganzfeld (e.g.,
Agrici et al., 2019;
Martin et al., 2021), specialized Maxwellian view (e.g.,
Cao et al., 2015;
Nugent & Zele, 2022), projector (
Allen et al., 2018;
Allen et al., 2019;
DeLawyer et al., 2020;
Hexley et al., 2020;
Spitschan et al., 2019;
Yamakawa et al., 2019), or display (
Blume et al., 2022;
Schöllhorn et al., 2023) system. In a review on stimulation devices used in the literature,
Conus and Geiser (2020) found that in most cases, the device had four or five primaries and was built from scratch using LEDs, optical bench components, and microprocessors (e.g., Arduino) with intensity controlled by pulse width modulation.
Also required for silent substitution are estimates of the photoreceptor action spectra of the observer, which, in humans, vary even among color-normal observers due to differences in ocular physiology. Before striking the retina, incident light must first travel through the lens and ocular media pigment of the observer, which act as prereceptoral filters to effectively shift the spectral sensitivity of the underlying photoreceptors. The crystalline lens of the eye accumulates its yellow pigment with age due to absorption of near-UV radiation (
Norren & Vos, 1974;
Pokorny et al., 1987;
Weale, 1988), and a yellower lens transmits less short-wavelength light. The macula pigment is a yellow carotenoid spot that sits above foveal photoreceptors and reduces the spectral sensitivity of underlying cones in a manner that becomes more pronounced for smaller stimulus field sizes (
Chen et al., 2001;
Whitehead et al., 2006). Observer age and stimulus field size therefore combine to alter the effective spectral sensitivity functions of a given observer under particular viewing conditions (
Figure 2; see also
Figures 5A,B).
The International Commission on Illumination (CIE) defines average colorimetric observer models with estimates of photoreceptor spectral sensitivities for a given age and field size. The CIE observers are based on decades of research involving predominantly psychophysical methods (
Smith & Pokorny, 1975;
Stiles & Burch, 1959;
Stockman et al., 1993,
1999;
Stockman & Sharpe, 2000;
Vos & Walraven, 1970) but also techniques such as suction electrode recording and microspectrophotometry of photoreceptors (e.g.,
Baylor et al., 1984;
Bowmaker et al., 1978;
Bowmaker & Dartnall, 1980). The
CIE 2006 Physiological Observer (developed in CIE 170-1:2006 and abbreviated here as CIEPO06:
CIE, 2006) defines fundamental cone spectral sensitivity functions for 2° and 10° field sizes and outlines a framework for calculating average spectral sensitivity functions for any age between 20 and 80 years and any field size between 1° and 10°. More recently, prompted by the pioneering research into melanopsin-containing ipRGCs (e.g.,
Al Enezi et al., 2011;
Berson et al., 2002;
Brown et al., 2013;
Dacey et al., 2005;
Gamlin et al., 2007;
Lucas et al., 2014;
Provencio et al., 2000;
Spitschan, 2019), the CIE released an International Standard—CIE S 026/E:2018 (
CIE, 2018)—that includes the melanopic and rhodopic action spectra alongside the CIEPO06 cone action spectra and outlines best practices for documenting photoreceptor responses to light.
With an accurately calibrated multiprimary system, one can predict
E(λ), the spectral output of the device (radiance or irradiance, in energy units), for any combination of primary gain settings using interpolation followed by a linear combination of primary spectra. Then, with photoreceptor action spectra appropriate to the intended observer, one can compute the activation level for each class of photoreceptors following the equation:
\begin{eqnarray}{E}_\alpha = \mathop \sum \limits_{380}^{780} E\left( \lambda \right){S}_\alpha \left( \lambda \right){\Delta }_\lambda \quad \end{eqnarray}
where
Eα is a scalar representing the photoreceptor activation for photoreceptor class
α (e.g., one of the five photoreceptors: S cones, M cones, L cones, rods, ipRGCs),
Sα(λ) is the estimated photoreceptor action spectrum, and Δ
λ is the width of the spectral sampling. By convention (
CIE, 2018;
Lucas et al., 2014), when this computation concerns the spectral sensitivity functions of retinal photoreceptors, the resulting measures may be called
α-opic irradiance.
Figure 3 shows an example spectral measurement alongside the
α-opic action spectra and the resulting
α-opic irradiance measures that follow from
Equation 1.
The goal of silent substitution is to invert this process to determine the device input settings to produce two spectra—a background spectrum and a modulation spectrum—where the resulting
α-opic quantities will be the same for all but the targeted photoreceptor(s). In the case where linearity holds across the entire
α-opic computation, silent substitution is typically computed using linear algebra, by inverting a fully or overdetermined set of equations describing the mapping from primaries to photoreceptor responses (e.g.,
Cao et al., 2015;
Maguire et al., 2016;
Shapiro et al., 1996). However, where linearity fails at some point in the system—for example, if the spectrum of a particular primary changes as a function of its intensity—then it may still be possible to obtain a solution using constrained numerical optimization (e.g.,
Spitschan et al., 2015).
In this Methods article, we introduce
PySilSub (
Martin et al., 2023), a Python toolbox that eases the computational burden of silent substitution by taking a structured approach to dealing with the various confounds and challenges outlined above. The toolbox provides flexible object-oriented support for multiprimary stimulation devices and individual colorimetric observer models, example calibration data for a range of devices, an intuitive interface for defining and solving silent substitution problems with linear algebra and constrained numerical optimization, and methods for accessing relevant standards and visualizing solutions. The code is actively maintained on GitHub (
https://github.com/PySilentSubstitution/pysilsub) under the MIT License and comes with a comprehensive testing suite and extensive online documentation with detailed examples. It is also registered with the Python Package Index (
https://pypi.org/project/pysilsub/) and can therefore be installed with the
pip packaging tool. Here, we provide an overview of the toolbox, describe key features, and demonstrate how it can be used.