Milestone 8 of 8

Make one controlled comparison

Change one configuration field, predict the effect, and verify that all unchanged alignment and conservation rules still hold.

The final run changes one declared setting. The purpose is not to find a favorable result; it is to show which evidence depends on which choice.

Goal

Run one controlled comparison by changing only the seed, checking count, batch size, or remainder policy, then explain the changes and the contracts that remain true.

Inputs

Start from the saved baseline configuration and replay result. Choose one setting and write down the old and new value before running the comparison. Keep the source file, schema revision, generator implementation, and all other configuration fields unchanged.

Examples:

  • change the seed and compare the resulting permutation;
  • change checking_count and compare group sizes and IDs;
  • change batch_size and compare batch boundaries;
  • change keep_remainder and compare the final batch and omitted IDs.

Do not change the source or edit the baseline manifest to make the comparison look consistent.

Deliverables

Record the baseline and comparison configurations, the changed setting, the resulting indexes, groups, batches, omitted IDs, and a short explanation of:

  • which artifacts must change;
  • which source identity and alignment contracts must continue to hold;
  • which differences are expected from the selected setting.

The comparison may be recorded with the project's split/batch evidence and report. Do not create an unrelated second project format.

Checks

Check that exactly one configuration field changed. Re-run the conservation, disjointness, uniqueness, ID-to-field alignment, batch-size, and remainder invariants for the comparison.

For a seed change, compare the recorded permutation rather than claiming that every pair of seeds must differ. For a count, size, or remainder change, confirm the corresponding group or batch effect and state any unaffected artifacts. Replay the baseline once more to ensure it was not overwritten.

Workspace

Keep the baseline and comparison evidence distinguishable under output/ and preserve the continuous workspace. The source and supplied manifests remain read-only; only generated evidence may differ.

Hints

HintWrite the hypothesis first
Before running, say which output should change and which invariants should remain true. This turns a second run into a comparison.
HintDo not overclaim a seed
A different seed usually changes a visible permutation, but the contract is the recorded result for the chosen fixtures, not a universal claim about every seed pair.

Review

Read the two runs side by side. Can another reader identify the one changed setting? Does the explanation distinguish a changed order from a broken alignment? Does it avoid calling the checking group a test set or making a probability claim?

How to check your work

The supplied fixture makes one setting change, records the expected effect, and reruns unchanged invariants. Compare the explanation with the artifacts, not just the final counts.