The use of mutation in testing experiments and its sensitivity to external threats

Akbar Siami Namin, Sahitya Kakarla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

34 Scopus citations

Abstract

Mutation analysts has emerged as a standard approach for empirical assessment of testing techniques. The test practitioners decide about cost-effectiveness of testing strategies based on the number of mutants the testing techniques detect. Though fundamental rigor to empirical software testing, the use of mutants in the absence of real-world faults has raised the concern of whether mutants and real faults exhibit similar properties. This paper revisits this important concern and disseminates interesting findings regarding mutants and whether these synthetic faults can predict fault detection ability of test suites. The results of controlled experiments conducted in this paper show that mutation when used in testing experiments is highly sensitive to external threats caused by some influential factors including mutation operators, test suite size, and programming languages. This paper raises the awareness message of the use of mutation in testing experiment and suggests that any interpretation or generalization of experimental findings based on mutation should be justified according to the influential factors involved.

Original languageEnglish
Title of host publication2011 International Symposium on Software Testing and Analysis, ISSTA 2011 - Proceedings
Pages342-352
Number of pages11
DOIs
StatePublished - 2011
Event20th International Symposium on Software Testing and Analysis, ISSTA 2011 - Toronto, ON, Canada
Duration: Jul 17 2011Jul 21 2011

Publication series

Name2011 International Symposium on Software Testing and Analysis, ISSTA 2011 - Proceedings

Conference

Conference20th International Symposium on Software Testing and Analysis, ISSTA 2011
CountryCanada
CityToronto, ON
Period07/17/1107/21/11

Keywords

  • experimental design
  • hand-seeded faults
  • mutants
  • mutation testing
  • real faults
  • statistical analysis

Fingerprint Dive into the research topics of 'The use of mutation in testing experiments and its sensitivity to external threats'. Together they form a unique fingerprint.

  • Cite this

    Namin, A. S., & Kakarla, S. (2011). The use of mutation in testing experiments and its sensitivity to external threats. In 2011 International Symposium on Software Testing and Analysis, ISSTA 2011 - Proceedings (pp. 342-352). (2011 International Symposium on Software Testing and Analysis, ISSTA 2011 - Proceedings). https://doi.org/10.1145/2001420.2001461