Milestone 7 of 8

Save and replay the system

Write configuration, data identity, predictions, traces, summaries, figure, runtime, and manifest evidence, then replay it separately.

The saved system should be understandable without the current editor state. Preserve the inputs, rules, traces, and runtime boundary before making a final claim.

Goal

Write configuration, data identity, predictions, neighbor and vote traces, summary, figure, runtime evidence, and manifest. Replay the system in a separate directory and record agreement or the first mismatch.

Inputs

Use all artifacts from the earlier milestones and the original files under data/. The manifest must identify the dataset and schema, configuration and k, source row order, output files, relevant Python/NumPy/Matplotlib versions, and implementation or fixture revision.

The replay starts from saved configuration and source identity and writes to a separate directory such as output/replay/. Never overwrite the original predictions or traces.

Deliverables

Extend src/main.py and src/report.py, or add a clearly separated replay function, to write:

  • output/neighbor_config.json;
  • output/predictions.csv;
  • output/neighbor_traces.json;
  • output/vote_traces.json;
  • output/checking_summary.json;
  • output/neighbors_and_queries.png;
  • output/run_manifest.json; and
  • a replay agreement or mismatch record under output/replay/.

The manifest should reference these files rather than copying every trace into one large record. The replay record compares source identity, query and reference order, configuration, neighbor IDs/distances, vote evidence, predictions, summary, and plot data.

Checks

Read each required artifact back and check its schema, identity, count, and links. Replay an unchanged system in a separate directory and compare every query's ordered neighbors, vote trace, prediction, summary, and plot data. Confirm that original files are unchanged.

Deliberately change one saved source row, k, or trace and check that the replay record names the first differing field or artifact. If the runtime is incompatible, record that limitation instead of claiming exact agreement.

Workspace

Keep original artifacts under output/ and replay artifacts under a distinct child directory. Keep output/run_manifest.json and the project-level report.md at stable paths throughout the project.

Hints

HintReplay from saved evidence
Do not use an unsaved table or a newly sorted array as replay input. Load the recorded source identity, configuration, and row order.
HintCompare identity before labels
Compare source and query IDs, then neighbor order and distances, then votes and predictions. The first mismatch narrows the cause.

Review

Open the original and replay directories side by side. Explain what replay agreement establishes for this fixture and why it does not establish future classification performance or a useful coordinate representation.

How to check your work

Checks compare the manifest and replay record with the supplied fixture. The supplied fixture keeps the original evidence immutable and makes a mismatch inspectable.