SyROCCo: Enhancing Systematic Reviews using Machine Learning

• DOI: 10.48550/arxiv.2406.16527
• Publication Date: 2024-06-24
• Authors: Zheng Fang; Miguel Arana‐Catania; Felix-Anselm van ...; Juliana Outes Vel...; Harry Bregazzi; Mara Airoldi; Eleanor Carter; Rob Procter
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Computation and Language; Computer Science - Digital Libraries; data mining; Information technology; T58.5-58.64; outcomes-based contracts; Machine Learning (cs.LG); evidence mapping; Computer Science - Computers and Society; machine learning; systematic review; JF20-2112; Computers and Society (cs.CY); Digital Libraries (cs.DL); Political institutions and public administration (General); Computation and Language (cs.CL)

The sheer number of research outputs published every year makes systematic reviewing increasingly time- and resource-intensive. This paper explores the use machine learning techniques to help navigate review process. ML has previously been used reliably 'screen' articles for - that is, identify relevant based on reviewers' inclusion criteria. application subsequent stages a review, however, such as data extraction evidence mapping, is in its infancy. We therefore set out develop series tools would assist profiling analysis 1,952 publications theme 'outcomes-based contracting'. Tools were developed following tasks: assign into 'policy area' categories; extract key information organisations, laws, geographical information; connect base an existing dataset same topic; subgroups may share thematic content. An interactive tool using these public with their have released. Our results demonstrate utility enhance accessibility within processes. These efforts show promise potentially yielding substantial efficiencies future broadening analytical scope. work suggests there be implications ease which policymakers practitioners can access evidence. While seem poised play significant role bridging gap between policy by offering innovative ways gathering, accessing, analysing from reviews, we also highlight current limitations need exercise caution application, particularly given potential errors biases.