Milestone 8 of 8

Write the experiment report

Explain one replay, one setting change, one failure, and the limits of the saved evidence in a short report.

The final report turns files into an explanation. It should describe what the runner proved, what it only observed, and what it cannot decide.

Goal

Write a short experiment report that explains one same-seed replay, one deliberate setting change, one failed run, and the limits of the evidence.

Inputs

Use the validated plan, at least three run directories, the root manifest, comparison artifacts, the replay record, and the source and runtime evidence. Read the actual CSV and JSON files rather than copying values from a console log. Include the supplied computation interface revision and the source dataset identity.

Deliverables

Complete report.md with these sections:

  1. Question and setup: state the small question, source identity, supplied computation, baseline, and selected configurations.
  2. Run evidence: summarize completed and failed runs, their statuses, and where their artifacts live.
  3. Comparison: state which setting changed and describe the observed differences without calling one result universally best.
  4. Replay: state whether the selected completed run agreed in a separate directory and name the compatible runtime boundary.
  5. Limits: explain what the evidence does not establish, including model quality, causal attribution, or future-runtime agreement.

Link each claim to a manifest, run, comparison, or replay artifact by stable path. Keep the prose short and use exact values when they matter.

Checks

Check that the report names a real baseline, includes a completed and a failed run, identifies the deliberate setting change, and records the replay result. Follow every cited path and confirm that status, configuration, and result match the prose.

Check that the report does not call the supplied computation a model, does not claim that one setting is best without a criterion, and does not turn a local browser check into a claim of private or secret evaluation. Confirm that the report distinguishes observation from interpretation.

Workspace

Keep the report at report.md. Do not duplicate all results in prose or move the source data into the report. The report is the reader's explanation of the durable artifacts; the manifest, run directories, comparison, and replay records remain the primary evidence.

Hints

HintWrite from the files
Open the manifest and one completed, failed, compared, and replayed artifact before drafting a conclusion.
HintSay what the evidence cannot show
A controlled setting change tells you what changed in this supplied computation. It does not establish a universal winner or a causal explanation.

Review

Read the report as a new reader. Can they reconstruct which run was baseline, what failed, what changed, and whether replay agreed? Remove adjectives that do not add evidence and replace broad claims with the exact boundary you tested.

How to check your work

Checks compare the report structure and claims with the supplied fixture. The supplied fixture is an exemplar of traceable writing, not a paragraph to copy without checking your own artifacts.