Abstract
Within neuroscience, psychology and neuroimaging, the most frequently used statistical approach is null-hypothesis significance testing (NHST) of the population mean. However, an alternative and equally valid question to ask about a population is how typical is the effect? To address this question, we infer an effect in each individual of the sample, and from that infer the prevalence of the effect in the population—i.e. the proportion of the population that would show the effect, if tested. We propose a novel Bayesian method to estimate such population prevalence that offers several advantages over population mean NHST. This method provides a population-level inference that is currently missing from study designs with small participant numbers, such as in traditional psychophysics and in precision neuroimaging. It delivers a quantitative Bayesian estimate of the prevalence of the effect in the population, with associated uncertainty, instead of reducing an experiment to a binary inference on a population mean. With simulations and examples from behavioral experiments, EEG and fMRI we show that the results obtained using population mean versus population prevalence can differ, particularly when effects are heterogenous across participants. We show how difference in prevalence can be directly estimated between and within groups, and how prevalence of different effect sizes can reveal a more detailed picture of the population. We argue that in many experimental applications in psychology and neuroscience, the individual participant is the natural replication unit of interest. However, within-participant results are usually viewed as fixed-effect case studies without a formal inference to the population. Bayesian prevalence provides this link. Bayesian prevalence is widely applicable to a broad range of studies in neuroscience, psychology and neuroimaging. Its emphasis on detecting effects within individual participants could also help address replicability issues in these fields.