A manual categorization of new quality issues on automatically-generated tests

Simple item page
dc.catalogadorjlo
dc.contributor.authorGalindo-Gutierrez, Geraldine
dc.contributor.authorNarea Carvajal, Maximiliano Agustín
dc.contributor.authorFernández, Blanco Alison
dc.date.accessioned2024-05-31T13:16:01Z
dc.date.available2024-05-31T13:16:01Z
dc.date.issued2023
dc.description.abstractDiverse studies have analyzed the quality of automatically generated test cases by using test smells as the main quality attribute. But recent work reported that generated tests might suffer from a number of quality issues not considered previously, thus suggesting that not all test smells have been identified yet. Little is known about these issues and their frequency within generated tests. In this paper, we report on a manual analysis of an external dataset consisting of 2,340 automatically generated tests. This analysis aimed at detecting new quality issues, not covered by past recognized test smells. We use thematic analysis to group and categorize the new quality issues found. As a result, we propose a taxonomy of 13 new quality issues grouped in four categories. We also report on the frequency of these new quality issues within the dataset and present eight recommendations that test generators may consider to improve the quality and usefulness of the automatically generated tests. As an additional contribution, our results suggest that (i) test quality should be evaluated not only on the tests themselves, but considering also the tested code; and (ii) automatically generated tests present flaws that are unlikely to be found in manually created tests and thus require specific quality checking tools.
dc.fuente.origenORCID
dc.identifier.doi10.1109/ICSME58846.2023.00035
dc.identifier.urihttps://doi.org/10.1109/ICSME58846.2023.00035
dc.identifier.urihttps://repositorio.uc.cl/handle/11534/86124
dc.information.autorucEscuela de Ingeniería; Narea Carvajal, Maximiliano Agustín; S/I; 1046125
dc.information.autorucEscuela de Ingeniería; Fernández, Blanco Alison; S/I; 1287130
dc.language.isoen
dc.nota.accesosin adjunto
dc.relation.ispartof2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)
dc.rightsregistro bibliográfico
dc.subject.ddc000
dc.subject.deweyCiencias de la computaciónes_ES
dc.titleA manual categorization of new quality issues on automatically-generated tests
dc.typecomunicación de congreso
sipa.codpersvinculados1046125
sipa.codpersvinculados1287130
sipa.trazabilidadORCID;2024-05-27
Files
Collections
Artículos de conferencia