Reproducible research artifacts

Start with a claim inventory. For each table, figure, or numerical assertion, name the inputs, execution command, raw outputs, analysis command, and expected result range. Use immutable identifiers for code, containers, datasets, and model checkpoints. A tag can move, while a digest identifies content under its hashing assumptions. Preserve configuration and dependency resolution, not merely a list of top-level libraries with unconstrained versions.

A manifest should also state hardware assumptions, peak memory, storage, approximate runtime, seeds, environment variables that change semantics, and licenses or access restrictions. Do not package credentials or private data into a public archive. If restricted inputs cannot be redistributed, document a legitimate access process and provide a synthetic smoke test with an explicit limit: it checks execution wiring, not reproduction of a sensitive-data result.

Provide a cheap installation check, a small correctness run, a reduced experiment, and the full reproduction path. State which claims each stage supports. A two-minute smoke test can verify imports and output schema but cannot validate a week-long convergence claim. Include expected logs and meaningful tolerances, especially when hardware changes affect numerical results. Separate setup failures from scientific disagreement so reviewers can tell whether the method ran as intended.

Keep raw observations and analysis scripts sufficient to regenerate reported summaries. If a figure uses medians across runs, publish the run identities and aggregation rule, including excluded failures and their reasons. Avoid manually edited spreadsheets as the only bridge from logs to a paper. Preserve units and missing-value handling in machine-readable outputs. A reproducible calculation on selectively reported data can still be misleading, so completeness of the evidence matters.

Ask someone outside the implementation team to run the documented path from a clean environment. Record unclear instructions and missing assumptions as artifact defects. Public availability, successful execution, and independent validation of the main result are separate accomplishments. ACM's artifact policy distinguishes related badges, while conference checklists ask authors to document experimental details and limitations. State precisely what review occurred instead of implying a stronger endorsement.

Archive a versioned release in a repository intended for durable access, with checksums and a citation identifier where available. Document how future readers can verify the archive and what may expire, such as model-hosting endpoints. Maintain an issue log for known deviations and corrected results. An artifact is most useful when it supports both reproduction and extension: modular experiments, documented interfaces, and clear boundaries let others test the next hypothesis without rebuilding the entire environment.