Replications in Human Factors research: Implications for education

Paul L. Derby, Keith S. Jones, Elizabeth A. Schmidlin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Replication research benefits the scientific community for two primary reasons. First, it demonstrates that a given finding can or cannot be repeated. This allows researchers to adjust their confidence in that finding accordingly (Hendrick, 1990). By doing so, replications allow scientific knowledge to be cumulative (Fisher, 1974; Kelly, 2006; Nickerson, 2000; Rosenthal, 1993). Second, replications sometimes demonstrate that a given finding does or does not generalize. Specifically, replications that employ different manipulations and procedures can demonstrate whether a given finding generalizes to different situations (Lindsay & Ehrenberg, 1993). This allows researchers to understand where a given finding might or might not be applicable. The current consensus suggests that replication research is uncommon outside of the "hard" sciences for a variety of reasons. Many believe that they lack the time, subjects, or funds necessary to reproduce original findings (Smith, 1970). Also, conducting a replication presents a risk that the study might produce different results (Lindsay & Ehrenberg, 1993). Lastly, by emphasizing the production of original findings, journal editors may discourage researchers from pursuing replications (Lindsay & Ehrenberg, 1993). If the consensus is correct, then there are two primary implications for the education and training of Human Factors professionals. First, Human Factors professionals should be taught to limit their confidence in a finding until they have established that it has been replicated. Second, Human Factors professionals should be taught more about the importance of replications and how to conduct them. However, we do not currently know how often replications are conducted in Human Factors. This study is the first to investigate the extent to which Human Factors professionals conduct replications. To do so, eight articles (hereafter referred to as parent articles) were selected from the 1991 issues of the journal, Human Factors. Each article that had referenced one of the eight parent articles between 1991 and September 2006 (hereafter referred to child articles) were also retrieved (N - 127). Two investigators coded and compared each child article against its 1991 parent article to determine whether the child article replicated its parent article. Our results indicated that the majority of the parent articles were replicated 1 to 10 times. In general, parent articles were more likely to be replicated when they had a large number of child articles. These replications were conducted despite the belief that social scientists rarely replicate previous research (Hendrick, 1990; Kelly, 2006; Lindsay & Ehrenberg, 1993; Schneider, 2004; Smith, 1970). This is a positive finding for two reasons. First, Human Factors professionals are replicating despite the constraints previously mentioned (Lindsay & Ehrenberg, 1993; Smith, 1970). Second, it implies that Human Factors professionals are being educated about the importance of replication research. However, there is one negative aspect of our findings. Our results revealed that not all original findings are being replicated. Therefore, Human Factors professionals should be taught to limit their confidence in previous research that has not been replicated.

Original languageEnglish
Title of host publication52nd Human Factors and Ergonomics Society Annual Meeting, HFES 2008
Pages672
Number of pages1
StatePublished - 2008
Event52nd Human Factors and Ergonomics Society Annual Meeting, HFES 2008 - New York, NY, United States
Duration: Sep 22 2008Sep 26 2008

Publication series

NameProceedings of the Human Factors and Ergonomics Society
Volume1
ISSN (Print)1071-1813

Conference

Conference52nd Human Factors and Ergonomics Society Annual Meeting, HFES 2008
CountryUnited States
CityNew York, NY
Period09/22/0809/26/08

Fingerprint Dive into the research topics of 'Replications in Human Factors research: Implications for education'. Together they form a unique fingerprint.

  • Cite this

    Derby, P. L., Jones, K. S., & Schmidlin, E. A. (2008). Replications in Human Factors research: Implications for education. In 52nd Human Factors and Ergonomics Society Annual Meeting, HFES 2008 (pp. 672). (Proceedings of the Human Factors and Ergonomics Society; Vol. 1).