Assessing data availability and research reproducibility in hydrology and water resources

• DOI: 10.1038/sdata.2019.30
• Publication Date: 2019-02-26T14:06:02Z
• Authors: James H. Stagge; David E. Rosenberg; Adel M. Abdallah; Hadia Akbar; Nour A. Attallah; Ryan James
• Keywords: Civil and Environmental Engineering; Statistics and Probability; 0301 basic medicine; Environmental Engineering; Reproducibility of Results; 028; hydrology; Library and Information Sciences; water resources; Civil Engineering; Article; 6. Clean water; Computer Science Applications; Education; 03 medical and health sciences; Engineering; Research Design; research reproducibility; Water Resources; Statistics, Probability and Uncertainty; Hydrology; data availability; Information Systems

AbstractThere is broad interest to improve the reproducibility of published research. We developed a survey tool to assess the availability of digital research artifacts published alongside peer-reviewed journal articles (e.g. data, models, code, directions for use) and reproducibility of article results. We used the tool to assess 360 of the 1,989 articles published by six hydrology and water resources journals in 2017. Like studies from other fields, we reproduced results for only a small fraction of articles (1.6% of tested articles) using their available artifacts. We estimated, with 95% confidence, that results might be reproduced for only 0.6% to 6.8% of all 1,989 articles. Unlike prior studies, the survey tool identified key bottlenecks to making work more reproducible. Bottlenecks include: only some digital artifacts available (44% of articles), no directions (89%), or all artifacts available but results not reproducible (5%). The tool (or extensions) can help authors, journals, funders, and institutions to self-assess manuscripts, provide feedback to improve reproducibility, and recognize and reward reproducible articles as examples for others.