Artifact packaging and independent reproduction

Map every main figure or table to input identifiers, preprocessing, code revision, configuration, run commands, raw outputs, and analysis scripts. Include licenses, permitted data access procedures, expected hardware, required services, and resource estimates. Pin dependencies by immutable versions or content identifiers where feasible, including model weights and generated intermediates. A container packages some userspace dependencies but not the host kernel, device driver, accelerator behavior, external service state, or a dataset that the image downloads later. Record those boundaries.

Treat result generation as a dependency graph. A final plot is reproducible only if its required ancestors are available or can be reconstructed under the documented assumptions. Include random streams, warmup, retries, failed runs, and filtering rules rather than only polished aggregate files. Explain unavoidable proprietary or restricted components and provide a lawful surrogate where possible, naming which claims it cannot validate. Checksums detect accidental byte disagreement against a trusted expected digest; they do not authenticate authorship if an attacker can replace both the file and the digest.

Provide a clean-environment setup, a small smoke test, and a full result-generation procedure. A smoke test can verify basic installation and data flow without reproducing the paper’s scale or statistical evidence. State expected durations, memory, storage, network access, and monetary costs before a reviewer starts. Provide oracles for task correctness and acceptable numeric variation. A command exiting successfully is insufficient if it loaded the wrong dataset or generated a cached reference plot instead of running the experimental treatment.

ACM-style artifact criteria distinguish public availability, evaluated functionality or reusability, and independently checked results. Use the definitions and version adopted by the specific venue, because terminology and review scope can differ. The ACM SIGSIM PADS 2024 criteria, for example, require a functional artifact to be documented, consistent, complete as feasible, and exercisable. An available archive alone does not establish those properties or validate every paper claim. Describe the actual checks performed and any inaccessible conditions rather than treating a badge as a blanket causal or correctness certificate.

Ask a fresh operator to follow the written instructions without undocumented help, then record interventions that were required. Preserve environment manifests and raw outputs from the independent execution. Distinguish rerunning authors’ code, rebuilding it from source, independently implementing the method, and testing a new platform; these establish different evidence. Compare correctness and uncertainty-aware outcomes rather than demanding byte-identical plots when the experiment permits variation. A discrepancy may expose a hidden dependency, a legitimate scope difference, or an error that should revise the claim.

The modular instrument gives a small manifest-sensitive transcript. Reset, set both m=16 and a=5, predict residue fifteen after three Steps, then Step three times. Separately Reset, set both m=16 and a=3, predict residue nine after three Steps, then Step three times. It starts at zero and adds a modulo m once per Step. This exact additive toy is not a hash, random generator, or artifact evaluator. It shows that omitting a configuration parameter can change an otherwise reproducible procedure’s result; matching one transcript does not establish a complete dependency inventory.