**Measurement efficiency is of concern when a large number of observations are required to obtain reliable estimates for parametric models of vision. The standard entropy-based Bayesian adaptive testing procedures addressed the issue by selecting the most informative stimulus in sequential experimental trials. Noninformative, diffuse priors were commonly used in those tests. Hierarchical adaptive design optimization (HADO; Kim, Pitt, Lu, Steyvers, & Myung, 2014) further improves the efficiency of the standard Bayesian adaptive testing procedures by constructing an informative prior using data from observers who have already participated in the experiment. The present study represents an empirical validation of HADO in estimating the human contrast sensitivity function. The results show that HADO significantly improves the accuracy and precision of parameter estimates, and therefore requires many fewer observations to obtain reliable inference about contrast sensitivity, compared to the method of quick contrast sensitivity function (Lesmes, Lu, Baek, & Albright, 2010), which uses the standard Bayesian procedure. The improvement with HADO was maintained even when the prior was constructed from heterogeneous populations or a relatively small number of observers. These results of this case study support the conclusion that HADO can be used in Bayesian adaptive testing by replacing noninformative, diffuse priors with statistically justified informative priors without introducing unwanted bias.**

*usefulness*of each stimulus in improving statistical inferences.

*usefulness*of each stimulus on a given trial is quantified by the expected reduction of entropy of the posteriors, or equivalently the uncertainty of the parameters. In general terms, the

*utility function*measures the usefulness of a given stimulus choice, written in the following form: where

*u*(

*s*,

*y*,

*θ*), called the sample utility, is a function of stimulus

*s*, observation

*y*, and parameter

*θ*;

*p*(

*y*|

*s*,

*θ*) is the statistical model; and

*p*(

*θ*) is the prior distribution of

*θ*. For example, a psychometric function

*p*(

*y*|

*s*,

*θ*) describes the probability of correct response for a given stimulus

*s*, with the parameter

*θ*containing a threshold and slope and the response

*y*being either a correct response or an incorrect response. The sample utility

*u*(

*s*,

*y*,

*θ*) quantifies the usefulness of stimulus

*s*with a specific parameter value

*θ*and a potential response

*y*. A particular specification of

*u*(

*s*,

*y*,

*θ*) is in which

*p*(

*θ*) is the prior distribution of

*θ*, and

*p*(

*θ*|

*y*,

*s*) is the posterior. Therefore,

*u*(

*s*,

*y*,

*θ*) can be interpreted as the reduction in the uncertainty about parameter

*θ*after a new trial with a stimulus

*s*and an observation

*y*. By taking the integral of the sample utility over all possible observations

*y*and parameters

*θ*, the derived

*expected utility U*(

*s*) in Equation 1 measures the

*expected information gain*brought by the stimulus

*s*(Cover & Thomas, 1991). The design that maximizes the expected utility is selected and presented in the next experimental trial. Hence, the optimal stimulus is expected to yield the largest information gain about the psychological function in the response on the next trial.

*p*(

*θ*), the prior distribution that represents the current state of knowledge about model parameters. For each trial during an adaptive testing session, the prior distribution is updated in a straightforward way by applying Bayes's rule with incoming data. However, the initial prior at the beginning of an experiment must be specified a priori by researchers. Commonly in Bayesian adaptive testing of visual functions, conservative priors—either uniform (Kontsevich & Tyler, 1999; Kujala & Lukka, 2006; Lesmes et al., 2006) or diffuse (Hou, Huang, Lesmes, Feng, Tao, Zhou, & Lu, 2010; Lesmes et al., 2010; Lesmes et al., 2015)—have been used.

*hierarchical adaptive design optimization*(HADO; Kim, Pitt, Lu, Steyvers, & Myung, 2014), which achieves even greater efficiency by applying informative priors constructed using data from observers who have previously conducted the same task. The improvement results from integrating hierarchical Bayesian modeling (HBM) with the standard entropy-based Bayesian adaptive testing procedures. In the work by Kim et al. (2014), HADO was applied to estimating the contrast sensitivity function (CSF) in a series of simulations with CSF data from 67 amblyopic and 80 healthy eyes, using as a baseline the quick CSF (qCSF) procedure (Lesmes et al., 2010), which embodies the standard Bayesian testing procedure to measure the CSF. A leave-one-out paradigm was used to compare HADO with qCSF by treating 146 subjects as being previously tested and the remaining subject as a new individual to be measured subsequently. The results showed that HADO achieved a decrease of around 2 dB (1 dB = 0.05 decimal log units) in the root-mean-square error (RMSE) in the estimation of CSF during the first 40 trials, and saved more than 20 trials to reach a 90% correct classification as an amblyopic eye versus a healthy eye, compared to the qCSF procedure, in a two-alternative forced-choice task.

*t*, the prior is expressed as

*n*indexes observers. The utility

*U*(

*s*) of a stimulus quantifies the expected reduction of entropy of parameters each stimulus can potentially bring, calculated by Equation 1. Then the optimal stimulus

_{t}*U*(

*s*) is administered and a new response

_{t}*t*+ 1. These steps repeat until a given number of trials are executed or a given criterion for accuracy and precision of estimation is reached.

**Figure 1**

**Figure 1**

*n*observers, a hierarchical Bayesian model can be formulated as

*p*(

*y*|

*s*,

*θ*) that describes how a response

_{i}*y*is generated given a stimulus

*s*and the

*i*th observer's parameter

*θ*. A middle-level model

_{i}*p*(

*θ*|

_{i}*η*), in the second line, defines the dependency among individual-level parameters

*θ*, conditional on the higher level parameter

_{i}*η*associated with the population. Depending on a researcher's assumption,

*p*(

*θ*|

_{i}*η*) may be modeled by a parametric distribution. For instance, if we assume that the CSFs of a population follow a Gaussian distribution

*N*(

*θ*|

_{i}*μ*, Σ), then

*η*would be the mean

*μ*and variance Σ. Alternatively, a nonparametric model such as a kernel density estimator can be specified if the underlying distribution is believed to deviate significantly from standard parametric distributions. The first line of Equation 3 specifies the prior distribution of the higher level parameter

*η*. When

*n*observers' worth of data are collected with observed responses

*y*

_{1:}

*(the subscript “1:*

_{n}*n*” denotes a collection of observations from a total of

*n*observers), the posterior distribution of the parameter

*η*is obtained by Subsequently, the prediction of the parameter

*θ*

_{n}_{+1}for a new observer is made by

*p*(

*θ*

_{n}_{+1}|

*y*

_{1:}

*) serves as an informative prior for the next observer*

_{n}*n*+ 1 in the experiment. It can be expected that with the increase in the number of collected observers

*n*,

*p*(

*θ*

_{n}_{+1}|

*y*

_{1:}

*) contains more information and therefore becomes more concentrated. On the other hand, when no prior data are available (i.e.,*

_{n}*n*= 0), HADO is reduced to the standard Bayesian adaptive testing procedure method with a noninformative, diffuse prior.

*γ*

^{max}, peak frequency

*f*

^{max}, bandwidth

*β*at half of the peak sensitivity, and low-frequency truncation level

*δ*). The SCF

*S*(

*f*) in Equation 6 is the reciprocal of the contrast threshold, corresponding to the level of contrast that is associated with a predefined performance level.

**Figure 2**

**Figure 2**

*p*(

*c*,

*f*) is the probability of generating a correct response at a specific contrast level

*c*and a spatial frequency

*f*,

*G*is the guess rate,

*L*is the lapse rate, Φ(·) is the cumulative Gaussian function, and

*σ*determines the slope of the psychometric function. An example of a cumulative Gaussian psychometric function is shown in Figure 2b. The guess rate is the probability of making a correct response when the contrast of stimuli approximates zero. In an

*N*-alternative, forced-choice (

*N*AFC) task, the guess rate is assumed to be equal to 1/

*N*. Figure 2b shows a guess rate of 0.1 for a 10AFC task (Hou et al., 2015). The lapse rate restrains the maximum probability of correct response to account for response errors caused by inattention. The slope of the psychometric function is preset to a value obtained from previous studies (Hou et al., 2015).

*f*(

*y*|

*c*,

*f*,

*γ*

^{max},

*f*

^{max},

*β*,

*δ*,

*G*,

*L*,

*σ*), mathematically describe how external stimulus variables

*c*and

*f*are transformed into underlying visual sensitivity, tuned by the parameters (

*γ*

^{max},

*f*

^{max},

*β*,

*δ*,

*G*,

*L*,

*σ*) specific to each observer being tested, and finally reflected as the probability of correct responses. The four parameters to be estimated in the present study are

*γ*

^{max},

*f*

^{max},

*β*, and

*δ*. The other three parameters

*G*,

*L*, and

*σ*are fixed as

*G*= 0.1,

*L*= 0.04, and

*σ*= 0.42 following Hou et al. (2015). Note that in a hierarchical modeling context, the model

*f*(

*y*|

*c*,

*f*,

*γ*

^{max},

*f*

^{max},

*β*,

*δ*) describes the individual-level data, corresponding to the third line in Equation 3.

*p*(

*θ*|

*η*) in Equation 3. In the following presentation, only the information relevant to the present study is described. For details of the baseline experiment, readers are directed to Hou et al. (2016).

^{2}, respectively. The order of the test blocks was L, L, M, H, LP, H. The first L condition was used for observers to dark-adapt and practice the qCSF test, and the two H conditions were included to assess the test–retest reliability of the qCSF method. In each test block, the qCSF procedure with a 10AFC letter-identification task was used to measure the CSF in 50 trials. Each observer finished the six blocks in approximately 70 min. For additional details about the procedure (i.e., use of a diffuse prior, adaptive stimuli selection, and Bayesian estimation), please refer to Hou et al. (2015).

*p*(

*θ*|

*η*) in Equation 3. To visualize these higher level distributions, we mapped the estimates of the four parameters onto two summary statistics of a CSF—the area under the log CSF (AULCSF; Applegate, Howland, Sharp, Cottingham, & Yee, 1997; Oshika, Okamoto, Samejima, Tokunaga, & Miyata, 2006) and the cutoff spatial frequency (cutSF; Huang, Tao, Zhou, & Lu, 2007; Zhou et al., 2006)—both of which are diagnostic measures of contrast sensitivity (Hou et al., 2010; Hou et al., 2015; Lesmes et al., 2010). The four-dimensional distributions of CSF parameters were thus transformed into a two-dimensional distribution of AULCSF and cutSF. Figure 3 shows the 75% equal-density contours of these distributions corresponding to the three conditions. Differences among the distributions are clearly visible in their locations in the parameter space, which are attributable to the experimental manipulations. Given that larger values of AULCSF and cutSF indicate better vision, the distribution of the CSFs in the H condition is located in the upper right in the space. Distributions representing the M and L conditions are located in regions covering smaller values of AULCSF and cutSF, exhibiting the expected ordering based on our luminance manipulations.

**Figure 3**

**Figure 3**

^{2}(measured by a Tektronix J17 photometer). A bit-stealing algorithm was used to achieve 9-bit grayscale resolution (Tyler, 1997). Observers viewed the display binocularly from a distance of 4 m in a dark room. A chin rest was used to help observers fix their head position relative to the screen. Two luminance conditions, H and L (the same as those in the baseline experiment), were tested.

**Figure 4**

**Figure 4**

*j*= 1, …, 50) by where

*Θ̂*

_{i}_{,}

*denotes the approximation of the true AULCSF of the*

_{T}*i*th observer estimated as already described in a given luminance condition, and

*Θ̂*

_{i}_{,}

*is the estimate of the AULCSF of the*

_{j}*i*th observer on the

*j*th trial. The constant 20 is multiplied to scale the results in decibel (dB) units, given that the parameter values are base-10 logarithms. Note that the RMSE reflects a combination of accuracy and precision in measurement theory (or equivalently, bias and variance in statistical inference) in a single summary statistic (Wackerly, Mendenhall, & Scheaffer, 2007). Accordingly, it can be considered an empirical instantiation of the mean squared error in the theory of point estimation in statistics (Lehmann & Casella, 1998).

**Figure 5**

**Figure 5**

**Figure 6**

**Figure 6**

^{1}We used the approximated true CSFs of the 10 observers (estimated from all responses with a diffuse prior) to generate simulated responses. For each CSF, Experiment 1 was executed 100 times with the simulated data. The RMSEs across all observers and replications were computed.

**Table 1**

*p*(

*θ*|

*η*) in Equation 3, was much more concentrated than the diffuse prior. In practice, however, it may not be feasible to collect that many observers before constructing an informative prior in HADO. Because a small sample contains limited information about its population, the prior constructed from the sample may still be somewhat diffuse, or potentially biased if there are outliers in the sample. The question of greatest interest is then how large the sample size should be in order to achieve sufficient efficiency, or whether a prior constructed from a relatively small sample will be even worse off. The goal of Experiment 2 was thus to investigate how the priors constructed from different sample sizes affect the estimation errors. The experiment measured CSFs of observers in the H luminance condition using the priors constructed from the data of different sample sizes in the same H condition. Therefore, the only possible source of measurement differences across conditions should be the amount of information in the priors.

*n*= 5, 12, and 30) from the 100 observers in the baseline experiment.

*n*= 5) can be quite effective in improving estimation.

**Figure 7**

**Figure 7**

**Table 2**

*θ*|

*a*,

*b*,

*σ*∼

*N*(

*a*+

*b*cov,

*σ*

^{2}), a normal distribution with its mean modeled by a linear regression with cov as the regressor. Plausibly, the cost of measuring these covariates could be very low. In the current study, individual variables such as age, gender, visual acuity, and eyeglass prescriptions are likely to covary with the CSF characteristics and are very easy to obtain. In that way, even more efficiency could be gained when these variables are included as covariates in a group-level model to obtain a more informative specific prior for new measurements.

**Disclosure**Luis Lesmes: Commercial Relationship(s), Adaptive Sensory Technology, Code I, E, P; Zhong-Lin Lu: Commercial Relationship(s), Adaptive Sensory Technology, Code I, P; Rest of authors: None

*Psychological Methods*, 9 (2), 250–271.

*Journal of the Optical Society of America A*, 11, 2375–2382.

*Journal of Refractive Surgery*, 14 (4), 397–407.

*Journal of Vision*, 14 (10): 157, doi:10.1167/14.10.157. [Abstract]

*Bayesian theory*. Chichester, UK: Wiley.

*Biometrika*, 83, 275–285.

*The American Statistician*, 39 (2), 83–87.

*Neural Computation*, 22 (4), 887–905.

*Elements of information theory*. New York: Wiley.

*An introduction to support vector machines*. Cambridge, UK: Cambridge University Press.

*Neural Computation*, 20 (3), 668–708.

*Neural Computation*, 23 (9), 2242–2288.

*Journal of the American Statistical Association*, 43 (241), 109–126.

*Investigative Ophthalmology & Visual Science*, 54 (12), 7266–7273.

*Optometry and Vision Science*, 78 (1), 56–64. [PubMed]

*Clinical applications of visual psychophysics*(pp. 70–106). Cambridge, UK: Cambridge University Press.

*International Ophthalmology Clinics*, 43 (2), 5–15.

*Transactions of the Ophthalmological Societies of the United Kingdom*, 99 (3), 391–397.

*Clinical applications of visual psychophysics*(pp. 11–41). Cambridge, UK: Cambridge University Press.

*Vision Research*, 47 (1), 22–34.

*Journal of Cataract & Refractive Surgery*, 15 (2), 141–148.

*Perception & Psychophysics*, 49, 227–229.

*The Annals of Mathematical Statistics, 29*

*,*41–59.

*Neural Computation*, 26, 2463–2492.

*Vision Research*, 34 (7), 885–912.

*Perception*, 36 (14), 1–16.

*Vision Research*, 39 (16), 2729–2737.

*Journal of Mathematical Psychology*, 50 (4), 369–389.

*Cognitive Science*, 30 (3), 1–26.

*Theory of point estimation*(2nd ed.). New York: Springer.

*Vision Research*, 46 (19), 3160–3176.

*Frontiers in Psychology*, 6, 1070, http://doi.org/10.3389/fpsyg.2015.01070.

*The Journal of the Acoustical Society of America*, 49, 467–477.

*Visual psychophysics: From laboratory to theory*. Cambridge, MA, MIT Press.

*Proceedings of the National Academy of Sciences, USA*, 102 (5), 1797–1802.

*Annals of Ophthalmology*, 13 (9), 1069–1071.

*Vision Research*, 46 (10), 1574–1584.

*Ophthalmology*, 113 (10), 1807–1812.

*Attention, Perception, & Psychophysics*, 28 (4), 377–379.

*Journal of the American Statistical Association*, 103, 1131–1154.

*Psychonomic Bulletin & Review*, 12 (4), 573–604.

*Multivariate density estimation: Theory, practice, and visualization*. New York: Wiley.

*Psychological Monographs*,

*74,*1–29.

*The Journal of the Acoustical Society of America*, 41, 782–787.

*Journal of the American Statistical Association*, 101, 1566–1581.

*Spatial Vision*, 10 (4), 369–377.

*Mathematical statistics with applications*. Belmont, CA: Cengage Learning.

*Bayesian evaluation of informative hypotheses*. (pp. 181–207). New York: Springer.

*Journal of Vision, 5*(9):6, 717–740, doi:10.1167/5.9.6. [PubMed] [Article]

*Perception & Psychophysics*, 33 (2), 113–120.

*Ophthalmology*, 87 (11), 1140–1149.

*Color science: Concepts and methods, quantitative data and formulae*(2nd ed.). New York: Wiley-Interscience.

*Vision Research*, 46 (5), 739–750.

^{1}The executable MATLAB programs of the simulations are available for download from http://faculty.psy.ohio-state.edu/myung/personal. Readers can also gain access to the source code after they sign a software license agreement with The Ohio State University.