Save the manifest and findings

The files and figure now exist, but a future reader still needs to know which source and schema produced them. This milestone records that identity and turns the measurements into a few evidence-backed observations.

Milestone goal

Write output/dataset_manifest.json and a short findings section that lets another reader identify the source, outputs, schema, counts, and report without reconstructing the project from code.

Manifest contract

Implement an interface such as:

The manifest must contain these top-level fields:

dataset_id, schema_version, source, outputs, field_contract,
status_counts, reason_counts, findings_file

Use the supplied dataset ID and schema revision from src/schema.py; do not invent a new identity. source records the raw filename and data-row count. outputs records each classified filename and row count. field_contract records the five supplied rules. The two count mappings agree with quality_summary.json. findings_file names the file containing the prose observations.

Keep the manifest JSON-compatible. Do not put NumPy scalar objects or arrays directly into it. A reader should be able to open it without importing your pipeline.

Findings contract

Write a short findings section in report.md with exactly three concise observations. Each observation must name its evidence, for example:

- Observation: 2 rows are missing readings.
  Evidence: status_counts.missing is 2; the rows are listed by source line in
  missing_records.csv.

The findings may describe counts, ranges, empty groups, visible gaps, or which records were excluded. They must not claim a cause that the file does not show, generalize beyond this supplied dataset, or call a value an outlier without a stated rule.

Deliverables

Save:

• output/dataset_manifest.json;
• the three findings in report.md;
• a read-back check for the manifest;
• the source and output identities used to create it.

Checks

Read the manifest and verify:

• dataset ID and schema revision match the supplied constants;
• source and output filenames are present and their counts match the files;
• field rules match the project contract;
• status and reason counts match quality_summary.json;
• the findings file exists and contains three evidence-linked observations;
• JSON read-back produces ordinary strings, numbers, booleans, lists, and mappings only.

Change one count in a temporary copy and ensure the audit can detect the disagreement. Do not repair the original artifacts by making the manifest hide the mismatch.

Review

Ask whether a reader who receives only the manifest, classified CSV files, and summary can identify the source, schema, counts, and report. Then read each finding and point to the exact row or summary field that supports it.

Next step

The final milestone reloads the artifacts in a fresh process, recomputes selected facts, and traces one plotted point and one rejection back to the source. The manifest is useful only if that audit can challenge it.

How to check your work

Checks compare the reference manifest's field names and count links. The findings remain your own description of the supplied fixture.