PY-99

Reproduce and Check a Saved Run

  • Medium–Hard
  • Run Reproduction
  • Python

Task

Write reproduce_saved_run(saved_run, source, workflow, make_isolated_location).

This problem uses small in-memory records to stand in for a saved experiment. The saved_run dictionary contains:

  • run_id, a non-empty name;
  • source_id and source_digest, identifying the input used by the saved run;
  • configuration, a dictionary of settings; and
  • artifacts, an ordered list of saved result records.

Each artifact record has a non-empty, unique relative path, a value, and a comparison of either "exact" or "close". A close comparison also has non-negative finite atol and rtol values. Exact artifacts use ordinary equality. Close artifacts use this rule for every numeric value:

abs(observed - expected) <= atol + rtol * abs(expected)

Lists and tuples must have the same length, dictionaries must have the same keys, and the comparison descends through those containers. A non-numeric value must match exactly even in a close artifact. A missing or extra artifact does not match.

source has source_id, digest, and records. The source identity is valid only when source["source_id"] == saved_run["source_id"] and source["digest"] == saved_run["source_digest"].

Call make_isolated_location(run_id) only after all saved-run and source validation succeeds. It returns a new location token. Then call workflow(configuration, records, location), passing a copy of the saved configuration and source records and the new location token. The supplied workflow returns:

{"location": location, "artifacts": [{"path": path, "value": value}, ...]}

The workflow must report the same location token it received. The location is evidence that this is a separate candidate run; do not compare results from a location chosen by the saved run.

Return this shape:

For an invalid source or saved-run contract, return "invalid", do not call either supplied helper, and use one of these errors: "invalid run record", "invalid source record", "source identity mismatch", or "invalid artifact contract". For a workflow result, compare the reported location first. Then visit saved artifacts in order. The first difference has this shape:

Use comparison: "location" when the workflow reports a different location. Use expected: None for a missing observed artifact and observed: None for an expected artifact that is absent. After the saved artifacts, an extra observed artifact is a difference. If there is no difference, return "reproduced"; otherwise return "mismatch". A mismatch is evidence about this run, not a claim about every run of the program.

Do not modify saved_run or source. Do not run a workflow before validation, create hidden global random state, print, or ask for input.

Example

The saved record might contain an exact list of IDs and a floating-point summary:

If the workflow returns the same IDs and means within the declared tolerance, the result is "reproduced". If it reports /runs/run-17 while the supplied isolated location is /candidate/run-17, report the location difference before looking at any artifact values.

Your implementation

Edit solution.py and keep this function signature:

You may import copy and math. Treat records as ordinary dictionaries and lists; no file format or external service is required. Keep the validation order and the first-difference order specified above.