PY-99
Reproduce and Check a Saved Run
Task
Write reproduce_saved_run(saved_run, source, workflow, make_isolated_location).
This problem uses small in-memory records to stand in for a saved experiment.
The saved_run dictionary contains:
run_id, a non-empty name;source_idandsource_digest, identifying the input used by the saved run;configuration, a dictionary of settings; andartifacts, an ordered list of saved result records.
Each artifact record has a non-empty, unique relative path, a value, and a
comparison of either "exact" or "close". A close comparison also has
non-negative finite atol and rtol values. Exact artifacts use ordinary
equality. Close artifacts use this rule for every numeric value:
abs(observed - expected) <= atol + rtol * abs(expected)
Lists and tuples must have the same length, dictionaries must have the same keys, and the comparison descends through those containers. A non-numeric value must match exactly even in a close artifact. A missing or extra artifact does not match.
source has source_id, digest, and records. The source identity is valid
only when source["source_id"] == saved_run["source_id"] and
source["digest"] == saved_run["source_digest"].
Call make_isolated_location(run_id) only after all saved-run and source
validation succeeds. It returns a new location token. Then call
workflow(configuration, records, location), passing a copy of the saved
configuration and source records and the new location token. The supplied
workflow returns:
{"location": location, "artifacts": [{"path": path, "value": value}, ...]}
The workflow must report the same location token it received. The location is evidence that this is a separate candidate run; do not compare results from a location chosen by the saved run.
Return this shape:
For an invalid source or saved-run contract, return "invalid", do not call
either supplied helper, and use one of these errors: "invalid run record",
"invalid source record", "source identity mismatch", or
"invalid artifact contract". For a workflow result, compare the reported
location first. Then visit saved artifacts in order. The first difference has
this shape:
Use comparison: "location" when the workflow reports a different location.
Use expected: None for a missing observed artifact and observed: None for
an expected artifact that is absent. After the saved artifacts, an extra
observed artifact is a difference. If there is no difference, return
"reproduced"; otherwise return "mismatch". A mismatch is evidence about
this run, not a claim about every run of the program.
Do not modify saved_run or source. Do not run a workflow before validation,
create hidden global random state, print, or ask for input.
Example
The saved record might contain an exact list of IDs and a floating-point summary:
If the workflow returns the same IDs and means within the declared tolerance,
the result is "reproduced". If it reports /runs/run-17 while the supplied
isolated location is /candidate/run-17, report the location difference before
looking at any artifact values.
Your implementation
Edit solution.py and keep this function signature:
You may import copy and math. Treat records as ordinary dictionaries and
lists; no file format or external service is required. Keep the validation
order and the first-difference order specified above.