September 2015
Volume 15, Issue 12
Vision Sciences Society Annual Meeting Abstract  |   September 2015
Human Perception of Statistical Significance and Effect Size
Author Affiliations
  • Bhavin Sheth
    Department of Electrical & Computer Engg, University of Houston Center for NeuroEngineering and Cognitive Science, University of Houston
  • Jasmine Patel
    Department of Biomedical Engg, University of Houston Center for NeuroEngineering and Cognitive Science, University of Houston
Journal of Vision September 2015, Vol.15, 337. doi:10.1167/15.12.337
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Bhavin Sheth, Jasmine Patel; Human Perception of Statistical Significance and Effect Size. Journal of Vision 2015;15(12):337. doi: 10.1167/15.12.337.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Statistics are ubiquitous in the information age. While they are fundamental to scientific discovery, understanding the layperson’s intuitive grasp of statistics is necessary if scientific advances and policies are to garner popular support. We asked two questions: First, how tightly matched are our perceptions of statistical significance and statistics? Second, are perceptual judgments of discriminability driven by statistical significance or by effect size? To address the first question, we displayed scatter plots of differently colored points chosen from two gaussian distributions had different mean/variance; observers judged if the two patterns of dots were similar or different. Perceptual judgments were compared to the results of statistical tests. Observers perceived the two arrangements of dots as “different” when there was no true difference at all (Type I: false positives), but were more conservative than statistics when the level of true discriminability increased (Type II: misses). When distributions overlap in space, we seem to be, by and large, immune to statistical significance. To address the second question, we displayed two plots in which we varied the level of statistical significance and effect size in opposite directions. Each plot contained red and blue points chosen from two Gaussian distributions of different mean and variance and observers had to judge, in a binary choice task, which of the two plots had red and blue points that appeared more different. The plots were designed so that one had the larger effect size (effect size is defined as the difference in means over the pooled standard deviation: Cohen’s d), and the other the higher level of statistical significance. Over a range of effect sizes and p-values, observes chose the plot with the larger effect size as being more different. Our results suggest that effect size, rather than statistical significance, drives perceptual judgments.

Meeting abstract presented at VSS 2015


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.