Best Reproducibility Advancement Award • SC24

The SC Best Reproducibility Advancement Award aims to recognize outstanding efforts in improving transparency and reproducibility of methods for high performance computing, storage, networking and analysis. This award will recognize the paper or a research object of a paper that most significantly advances the state of art in transparency and reproducibility.

In this context, a research object refers to a collection of computational artifacts that extend beyond the submitted paper, such as software, datasets, environment configuration, mechanized proofs, benchmarks, or test suites with scripts.

Goals

This award aims to encourage transparency and reproducibility and to reward outstanding efforts that broadly fall under the following scope:

Improvements in transparency and reproducibility of research in HPC, e.g., via tools or methods
Efforts to build transparent and reproducible research objects
Experimental design methodologies for verifying and future-proofing scientific claims and conclusions
Generation and publication of open datasets/software of relevance to the SC community
General advancements in transparency and reproducibility

Criteria

The award will be given based on the advancements in transparency and reproducibility and/or extraordinary examples of reproducible research in a paper or research object. Typical examples include:

Developing novel tools and methodologies to assist transparency and reproducibility
Demonstrating and studying transparency and reproducibility at SC
Enabling transparency and reproducibility in a subfield at SC with barriers for transparency and reproducibility
Showcasing best practices
Open sourcing research objects

Packaging research objects in a novel/particularly helpful way
Building research objects tailored to education
Automating reproduction
Adapting research objects for alternative experiment environments
Including original measurements and information about the experiments

Nomination Process

All accepted SC papers with AD/AE are eligible for this award and can be nominated by the SC Technical Paper Committee or SC Artifact Description/Evaluation Committee as part of the usual review process.

Each submission will be given the opportunity to highlight their advancement in transparency and reproducibility as part of their submission form. This information would then be considered during the selection process.