Milestone 7 of 8

Save and replay the investigation

Write configuration, data, full search evidence, selection, figure, runtime, and manifest artifacts, then replay them separately.

An investigation becomes easier to trust when its inputs, choices, outputs, and runtime boundary can be loaded again. Save the evidence before writing the conclusion.

Goal

Write the configuration, generated data, full search evidence, selected result, figure, runtime information, and manifest. Replay the investigation in a separate output directory and record agreement or the first mismatch.

Inputs

Use all artifacts from the earlier milestones and the original source identity. The manifest must identify the configuration, generated data, search scores, selected result, figure, report, relevant Python/NumPy/Matplotlib versions, and the generator or implementation revision used.

The replay starts from saved configuration and source identity, not from the current editor state. Write it to a separate directory such as output/replay/; do not overwrite the original evidence.

Deliverables

Extend src/main.py and src/report.py or add a clearly separated replay function to write:

  • output/experiment_config.json;
  • output/generated_data.csv;
  • full-grid search evidence;
  • output/selected_result.json;
  • output/fit_and_differences.png;
  • output/run_manifest.json; and
  • a replay agreement or mismatch record in output/replay/.

The manifest should reference files rather than copy every array into one large object. The replay record compares input, variation, observed values, score grid, selected result, summaries, and plot data. If the runtime is not compatible, record that limitation instead of claiming exact replay.

Checks

Read every required artifact back and check its identity, schema, row count, and selected values. Replay the unchanged configuration in a separate directory and compare arrays, grid scores, selected parameters, summaries, and plot data. Confirm that the original output remains unchanged.

Deliberately change one saved input or selected artifact and confirm that the replay record names the first differing field or file. Do not reduce every mismatch to a generic failure. Check that a missing or stale artifact is not described as an agreement.

Workspace

Keep the original artifacts under output/ and replay artifacts under a distinct child directory. Keep the manifest at output/run_manifest.json and the report at the project-level report.md, using these stable paths throughout the project.

Hints

HintManifest paths are part of the result
A manifest that lists only a score cannot lead another reader to the input and code that produced it. Record source, configuration, runtime, and artifact identity.
HintCompare in causal order
Compare source and configuration first, then generated arrays, the score grid, selected result, and plot data. The first disagreement narrows the cause.

Review

Open the original and replay directories side by side. Explain what same-run agreement establishes under the recorded runtime and why it does not promise agreement under every future dependency version.

How to check your work

Checks compare the manifest and replay record with the supplied fixture. The supplied fixture keeps original evidence immutable and makes a mismatch inspectable.