Abstract
The use of statistics has proliferated over the years so that research is now interpreted on the basis of p-values, with significance threshold sociologically determined as a p-value of 0.05 or less. While researchers commonly use statistical software to test for significance, human perception of significance has yet to be studied. Identifying our intuitions of significance helps one understand rapid-fire human perceptual judgments of high volume data (signal) embedded in noise, and has the potential to uncover our subjective biases and therefore, be a launching board toward removing these statistical "blind spots". Above all, it is a window into the true threshold of statistical significance in the human brain. Here, we examine the ability of adult humans to visually discriminate between two normal probability distributions that have i) identical means but differing variance: the variance of the test distribution on a given trial being 1.21, 1.44 or 3.24, and ii) identical variance but differing means: the mean of the test distribution on a given trial being 0, 0.1, or 0.3. In both experiments, the standard distribution had zero mean and unit variance. On a given trial, 2000 points (equally chosen from the two distributions) were displayed in two different colors. The observer had to judge, in a binary choice paradigm, if the two colors were similarly or differently arranged. On each trial, we calculated the p-value of the null hypothesis using Kolmogorov-Smirnoff statistics. Observers (N = 15) were less likely than statistical tests to reject the null hypothesis, i.e. to judge the two distributions as different. Even on trials in which the two distributions were highly discriminable (p-value <0.0001), observers did not judge them as such on i) 48% and ii) 14% of trials, on average. Further analyses and experiments will test sensitivity to p-value.
Meeting abstract presented at VSS 2012