Repeatability Evaluation

Authors of accepted Regular, Short, and Tool papers are eligible and encouraged, but not required, to participate. Tool papers are specifically encouraged to submit artifacts. The artifact process begins only after paper notification. Submission and notification dates remain tentative, and the separate artifact portal is not yet open. Badge terminology, final platform details, and exceptional arrangements will be announced before that process opens.

Submit one ZIP archive no larger than 600 MB. It must include an artifact-description PDF that identifies the included results, explains the system requirements, gives complete installation and execution instructions, describes the evaluation scenarios and expected outputs, and maps those outputs to the corresponding paper figures, tables, or claims. Author names, affiliations, repository identities, and account names may remain visible. Explain any expected variation caused by stochastic behavior.

Artifact packages are not anonymous: author names, affiliations, repository identities, account names, and ordinary file metadata may be visible to evaluators. Evaluator identities remain confidential to authors. Packages remain confidential review material; disable or clearly disclose analytics that could reveal evaluator identities.

A self-contained Docker image or OVA/OVF virtual machine is strongly encouraged. Software should run on a major operating system, or the archive should include the environment needed to run it. The package should run without network access when practicable; authors whose artifact requires licensed software, special hardware, unusually large data, or exceptional computing resources should follow the alternative-arrangement instructions in the final call. Packages that do not satisfy the final artifact instructions may be declined without evaluation.