Evaluating classifiers in SE research: the ECSER pipeline and two replication studies

• DOI: 10.1007/s10664-022-10243-1
• Publication Date: 2022-11-08T16:11:09Z
• Authors: Davide Dell’Anna; Fatma Başak Aydemir; Fabiano Dalpiaz
• Keywords: Replication

Abstract Context Automated classifiers, often based on machine learning (ML), are increasingly used in software engineering (SE) for labelling previously unseen SE data. Researchers have proposed automated classifiers that predict if a code chunk is clone, requirement functional or non-functional, the outcome of test case non-deterministic, etc. Objective The lack guidelines applying and reporting classification techniques research leads to studies which important steps may be skipped, key findings might not identified shared, readers find reported results (e.g., precision recall above 90%) credible representation performance operational contexts. goal this paper advance ML4SE by proposing rigorous ways conducting research. Results We introduce ECSER (Evaluating Classifiers Software Engineering Research) pipeline, includes series evaluating SE. Then, we conduct two replication where apply recent requirements testing. Conclusions In addition demonstrating applicability demonstrate ’s usefulness: only do confirm strengthen some original authors, but also discover additional ones. Some these contradict