Abstract
As a medium for conducting behavioral experiments, the internet offers the opportunity to collect large samples from a broad cross-section of the population on a relatively low budget. Despite the increasing use of the internet as a means of gathering data for psychology experiments, it is unclear how comparable internet-based data is to data gathered in the lab. Furthermore, it is not clear how recruitment method might impact data quality: for instance, tests conducted on the internet might produce comparable results to tests conducted in the lab, as long as the participants were recruited through traditional methods (i.e. privately). In order to assess the quality of data from internet-based experiments, we compared data from the Cambridge Face Memory Test (Duchaine & Nakayama, 2006) from participants tested in the lab and on the web, with different recruitment methods. Specifically, data were gathered from (a) 3004 unpaid participants who followed links to our website (testmybrain.org) to ‘test their skills’ (public/internet), (b) 594 participants recruited through the Australian twin registry, via traditional methods, but tested on the internet at testmybrain.org (private/internet), and (c) 209 participants tested in the lab (private/lab). Reliability, as measured by cronbach's alpha, was similar across all three datasets (public/internet: 0.90; private/internet: 0.89; private/lab: 0.89). Performance, in terms of proportion correct, was also comparable in the three datasets (public/internet: 0.76, SD = 0.13; private/internet: 0.74, SD =0.14; private/lab: 0.72, SD = 0.13). Our data indicate that, even for tests like the Cambridge Face Memory Test that include complex visual stimuli (faces), the internet has the potential to provide data comparable to data gathered in the lab and from participants recruited through more traditional methods.