Purchase this article with an account.
John Hayes, Adam Preston, James Sheedy; Graphical comparison of means in within subject designs. Journal of Vision 2010;10(7):1315. doi: 10.1167/10.7.1315.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
In vision sciences it is common to have study designs with many within subject conditions. The data are often presented in bar graphs with standard error bars. Non-overlapping standard error bars do not necessarily mean statistically significant difference. Similarly, overlapping 95% confidence intervals do not necessarily mean lack of a significant difference. We reviewed the literature that suggests a confidence interval can be derived that allows comparison between all means on a single chart. We then provide a simple graphical method in Excel that uses stacked bar graphs to create 84% confidence intervals in which non-overlapping bars are significant at an unadjusted p<.05. The bars can also be constructed using a Bonferonni adjustment considering (n-1)! comparisons. As an example we provide the results of a reaction time detection task using PowerPoint slides with 8 emphasis conditions (Bold, Italic, Underline, CAPS, red, yellow, green, blue) on 3 backgrounds (white, black, dark blue). We propose a method of estimating the standard error from the output of a maximum likelihood mixed model analysis of variance (SPSS 17, IBM Corp). Specifically we perform a one way analysis of variance and use the largest standard error of the differences (SED) from the paired comparisons output. The SEM is estimated as the square root of SED2/2. SEDs vary across comparisons if there is missing data, so our estimate uses the largest value. The degrees of freedom are n-1. The non-overlapping bars are virtually identical to the ANOVA paired comparisons. Because our example had heterogeneity of variance across groups, we also estimated the means and standard errors directly with a Bayesian Monte Carlo Markov Chain analysis (AMOS 18, IBM Corp) and computed the same confidence interval. In this example both analyses provided very similar results.
This PDF is available to Subscribers Only