Observatorio de I+D+i UPM

Memorias de investigación
Book chapters:
Challenges of Evaluating the Quality of Software Engineering Experiments!
Year:2013
Research Areas
  • Information technology and adata processing
Information
Abstract
Good-quality experiments are free of bias. Bias is considered to be related to internal validity (e.g., how well experiments are planned, designed, executed, and analysed). Quality scales and expert opinion are two approaches for assessing the quality of experiments. Aim: Identify whether there is a relationship between bias and quality scale and expert opinion predictions in SE experiments. Method: We used a quality scale to determine the quality of 35 experiments from three systematic literature reviews. We used two different procedures (effect size and response ratio) to calculate the bias in diverse response variables for the above experiments. Experienced researchers assessed the quality of these experiments. We analysed the correlations between the quality scores, bias and expert opinion. Results: The relationship between quality scales, expert opinion and bias depends on the technology exercised in the experiments. The correlation between quality scales, expert opinion and bias is only correct when the technologies can be subjected to acceptable experimental control. Both correct and incorrect expert ratings are more extreme than the quality scales. Conclusions: A quality scale based on formal internal quality criteria will predict bias satisfactorily provided that the technology can be properly controlled in the laboratory.
International
Si
10.1007/978-3-642-37395-4_11
Book Edition
1
Book Publishing
Springer Berlin Heidelberg
ISBN
978-3-642-37394-7
Series
Book title
Perspectives on the Future of Software Engineering
From page
159
To page
177
Participants
  • Autor: Oscar Dieste Tubio (UPM)
  • Autor: Natalia Juristo Juzgado (UPM)
Research Group, Departaments and Institutes related
  • Creador: Grupo de Investigación: Ingeniería del Software
S2i 2019 Observatorio de investigación @ UPM con la colaboración del Consejo Social UPM
Cofinanciación del MINECO en el marco del Programa INNCIDE 2011 (OTR-2011-0236)
Cofinanciación del MINECO en el marco del Programa INNPACTO (IPT-020000-2010-22)