An evaluation of mutation and data-flow testing: A meta-analysis

Sahitya Kakarla, Selina Momotaz, Akbar Siami Namin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Scopus citations

Abstract

Mutation testing is a fault-based testing technique for assessing the adequacy of test cases in detecting synthetic faulty versions injected to the original program. The empirical studies report the effectiveness of mutation testing. However, the inefficiency of mutation testing has been the major drawback of this testing technique. Though a number of studies compare mutation to data flow testing, the summary statistics for measuring the magnitude order of effectiveness and efficiency of these two testing techniques has not been discussed in literature. In addition, the validity of each individual study is subject to external threats making it hard to draw any general conclusion based solely on a single study. This paper introduces a novel meta-analytical approach to quantify and compare mutation and data flow testing techniques based on findings reported in research articles. We report the results of two statistical meta-analyses performed on comparing and measuring the effectiveness as well as efficiency of mutation and data-flow testing based on relevant empirical studies. We focus on the results of three empirical research articles selected from the premier venues with their focus on comparing these two testing techniques. The results show that mutation is at least two times more effective than data-flow testing, i.e., odds ratio= 2.27. However, mutation is three times less efficient than data-flow testing, i.e., odds ratio= 2.94.

Original languageEnglish
Title of host publicationProceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011
Pages366-375
Number of pages10
DOIs
StatePublished - 2011
Event4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011 - Berlin, Germany
Duration: Mar 21 2011Mar 25 2011

Publication series

NameProceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011

Conference

Conference4th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2011
CountryGermany
CityBerlin
Period03/21/1103/25/11

Keywords

  • Data flow testing
  • Meta-analysis
  • Mutation testing
  • Statistical analysis

Fingerprint Dive into the research topics of 'An evaluation of mutation and data-flow testing: A meta-analysis'. Together they form a unique fingerprint.

Cite this