Milestone 8 of 8

Write and audit the report

Connect every statement to saved item-level or aggregate evidence and state undefined values and limitations plainly.

The report should make the evaluation definitions and evidence easy to follow. Every claim should point to an item-level or aggregate artifact.

Goal

Write and audit a concise report that states the definitions, totals, per-label and slice evidence, paired differences, undefined values, and limitations.

Inputs

Use aligned_items.csv, overall summaries, both confusion tables, per_label_results.csv, slice_results.csv, paired_comparison.csv, category counts, all figures, evaluation_manifest.json, and the replay record. Read the saved files rather than relying on console output.

Deliverables

Complete the project-level report.md with these sections:

  1. Question and alignment: state the supplied records, ID join, configured label order, and why row position was not used.
  2. Definitions and totals: explain match fraction, confusion-table axes, per-label counts and ratios, and null for undefined denominators.
  3. Slices and paired comparison: state denominators, describe one source group, and explain the four paired outcome categories.
  4. Evidence: trace one item through aligned fields and paired category, and point to the tables and shared-scale figures.
  5. Replay and limits: state agreement or mismatch and explain what this fixture cannot establish about significance, fairness, causality, or future performance.

Audit every numerical statement against a saved artifact. Use “match fraction for these supplied items” rather than “accuracy” unless a later subject has defined that term for a different boundary.

Checks

Check that the report names a real item, follows its labels through the aligned record and paired comparison, and agrees with overall, per-label, slice, and category artifacts. Confirm that undefined ratios remain visible as undefined and that replay claims match the replay record.

Check that the report states at least one concrete limitation and separates observations from hypotheses. Remove claims about ranking, significance, causes, fairness, model selection, or future performance that the project does not test.

Workspace

Keep the report at the project-level report.md. Do not duplicate every table or trace in prose. The report explains durable files; the manifest, aligned records, tables, figures, paired comparison, and replay record remain the primary evidence.

Hints

HintFollow one item
Start with an item ID, read its expected and two predicted labels, then find its table cells and paired category before explaining an aggregate.
HintUse the supplied names
A measure called match_fraction has a precise local meaning. Do not silently rename it to a broader claim.

Review

Read the report as a new reader. Can they tell how rows were aligned, where a ratio came from, what a slice denominator was, which items differed, and what the evidence leaves unknown? Replace broad wording with an artifact-backed statement or remove it.

How to check your work

Checks compare the report structure and evidence links with the supplied fixture. The supplied fixture is an exemplar of traceable writing, not a conclusion to copy without checking your own artifacts.