Applying Inter-Rater Reliability and Agreement in collaborative Grounded Theory studies in software engineering

• DOI: 10.1016/j.jss.2022.111520
• Publication Date: 2022-10-01T03:29:43Z
• Authors: Jessica Díaz; Jorge Pérez; Carolina Gallardo; Ángel González-Pr...
• Keywords: Inter-Rater Reliability; Trustworthiness

The qualitative research on empirical software engineering that uses Grounded Theory is increasing (GT). trustworthiness, rigor, and transparency of GT data analysis can benefit, among others, when multiple analysts juxtapose diverse perspectives collaborate to develop a common code frame based consensual consistent interpretation. Inter-Rater Reliability (IRR) and/or Agreement (IRA) are commonly used techniques measure consensus, thus shared However, minimal guidance available about how IRR/IRA during the iterative process GT, so researchers have been using ad hoc methods for years. This paper presents systematically measuring in studies, appropriate, which grounded previous systematic mapping study collaborative field engineering. Meta-science guided us analyze issues challenges formalize GT. guides incrementally generate theory while ensuring consensus constructs support it, improving transparency, promoting communicability, reflexivity, replicability research. application this seems its feasibility. In absence further confirmation, would represent first step de facto standard be applied those studies may benefit from techniques.