Memorias de investigación
Communications at congresses:
The Risk of Using the Q Heterogeneity Estimator for Software Engineering Experiments
Year:2011
Research Areas
  • Information technology and adata processing
Information
Abstract
Background: All meta-analyses should include a heterogeneity analysis. Even so, it is not easy to decide whether a set of studies are homogeneous or heterogeneous because of the low statistical power of the statistics used (usually the Q test). Objective: Determine a set of rules enabling SE researchers to find out, based on the characteristics of the experiments to be aggregated, whether or not it is feasible to accurately detect heterogeneity. Method: Evaluate the statistical power of heterogeneity detection methods using a Monte Carlo simulation process. Results: The Q test is not powerful when the meta-analysis contains up to a total of about 200 experimental subjects and the effect size difference is less than 1. Conclusions: The Q test cannot be used as a decision-making criterion for meta-analysis in small sample settings like SE. Random effects models should be used instead of fixed effects models. Caution should be exercised when applying Q test-mediated decomposition into subgroups.
International
No
Congress
ESEM - International Symposium on Empirical Software Engineering and Measurement
960
Place
Banff, Alberta (CANADA)
Reviewers
Si
ISBN/ISSN
978-1-4577-2203-5
Start Date
19/09/2011
End Date
23/09/2011
From page
68
To page
76
ESEM 2011 Fifth International Symposium on Empirical Software Engineering and Measurement
Participants
  • Autor: Oscar Dieste Tubio UPM
  • Autor: Enrique Fernández Universidad Nacional de La Plata. Buenos Aires, Argentina
  • Autor: Ramón García-Martínez Universidad Nacional de Lanus. Buenos Aires, Argentina
  • Autor: Natalia Juristo Juzgado UPM
Research Group, Departaments and Institutes related
  • Creador: Grupo de Investigación: Ingeniería del Software