Observatorio de I+D+i UPM

Memorias de investigación
Cursos, seminarios y tutoriales:
Replication of Software Engineering Experiments
Áreas de investigación
  • Informática
Replication is one of the key mechanisms of the experimental paradigm. Through replication it is possible to verify the findings observed in earlier experiments. Campbell and Stanley explain the role of replication as "The experiments we do today, if successful, will need replication and cross-validation at other times under other conditions before they can become an established part of science, before they can be theoretically interpreted with confidence". The immaturity still plaguing the experimental paradigm applied to SE (most likely a consequence of its short lifetime) has been an obstacle to replications. At present, any change in a replication, no matter how small, can lead to alterations in the experiment outcomes that are hard (if not impossible) to interpret. Non-systematic changes to the experiment components cause the replication results to be highly variable ruling out the combination of results. Corroboration of the previous experiment findings was impossible. This state of affairs regarding SE experiment replications has caused some disagreement with respect to how previously observed findings should be verified. Some recommend reusing the baseline experiment materials to run replications and thus assure that the replications are similar enough for their results to be comparable. Others recommend using methods and materials not used in the baseline experiment to run replications and thus confirm that the results of the replications are not dependent on local experimental conditions. This tutorial discusses in depth the concept of replication. We will learn there are different types of replications, all of which are necessary to verify different aspects of an experiment. We will also see which are all those experiment aspects that need to be double checked. Some questions all SE experimenters performing replications wonder (How close a result need to be to consider it similar, Which replication is priority after this one, How many more replications I need to do and which ones, etc.) will be discussed.
Nombre congreso
LASER Summer School: Empirical Software Engineering
Entidad organizadora
Chair of Software Engineering - ETH Zurich
Nacionalidad Entidad
Lugar/Ciudad de impartición
Isla Elba, Italia
Fecha inicio
Fecha fin
Esta actividad pertenece a memorias de investigación
  • Autor: Natalia Juristo Juzgado (UPM)
Grupos de investigación, Departamentos, Centros e Institutos de I+D+i relacionados
  • Creador: Grupo de Investigación: Ingeniería del Software
S2i 2021 Observatorio de investigación @ UPM con la colaboración del Consejo Social UPM
Cofinanciación del MINECO en el marco del Programa INNCIDE 2011 (OTR-2011-0236)
Cofinanciación del MINECO en el marco del Programa INNPACTO (IPT-020000-2010-22)