Wanted: standards for automatic reproducibility of computational experiments

• DOI: 10.48550/arxiv.2307.11383
• Publication Date: 2023-01-01
• Authors: Grayson, Samuel; Milewicz, Reed; Teves, Joshua; Katz, Daniel S.; Marinov, Darko
• Keywords: Software Engineering (cs.SE); FOS: Computer and information sciences; Computer Science - Software Engineering

Submitted to SE4RS'23 Portland, OR<br/>Those seeking to reproduce a computational experiment often need to manually look at the code to see how to build necessary libraries, configure parameters, find data, and invoke the experiment; it is not automatic. Automatic reproducibility is a more stringent goal, but working towards it would benefit the community. This work discusses a machine-readable language for specifying how to execute a computational experiment. We invite interested stakeholders to discuss this language at https://github.com/charmoniumQ/execution-description .<br/>