Milestone 8 of 8

Write and audit the corpus report

Explain the corpus boundary, decoding and line rules, measurements, findings, usable records, replay, and limitations from saved evidence.

The report turns inspection artifacts into a careful account of what the corpus contains. Every statement should lead back to a saved record.

Goal

Write and audit a concise corpus report that explains the boundary, decoding and line rules, measurements, findings, examples, limitations, and which records remain usable.

Inputs

Use document_inventory.csv, document_measurements.csv, corpus_summary.json, source_group_summary.csv, inspection_findings.csv, selected_examples.json, duplicate_content_groups.json, all figures, inspection_manifest.json, and the replay record. Read the saved files rather than relying on console output.

Deliverables

Complete the project-level report.md with these sections:

  1. Corpus boundary: state manifest identity, file inventory, byte-first checks, UTF-8 rule, and accepted/rejected accounting.
  2. Line and measurement rules: explain LF, CRLF, lone CR, normalized analysis text, line definitions, and code-point measurements.
  3. Findings and examples: describe configured rules, final states, one traceable finding, one escaped example, and duplicate-byte evidence.
  4. Numerical and visual evidence: state corpus or source-group counts and two observations tied to document IDs or saved tables and figures.
  5. Replay and limits: state agreement or mismatch, which records remain usable, and what the inspection cannot establish about language, quality, representativeness, or model suitability.

Audit every numerical statement against a saved artifact. Use “this fixture contains” for observations and avoid turning a code-point count into a claim about words, tokens, meaning, or language quality.

Checks

Check that the report names the manifest and follows one accepted and one rejected or flagged record through their artifacts. Confirm that line-boundary counts, measurement totals, final-state counts, finding evidence, example locations, figure observations, and replay claims agree with saved files.

Check that the report states at least one concrete limitation and separates observations from hypotheses. Remove claims about language, quality, representativeness, privacy, or model suitability that the project does not test.

Workspace

Keep the report at the project-level report.md. Do not duplicate every measurement or excerpt in prose. The report explains durable files; the inventory, measurements, findings, examples, figures, manifest, and replay record remain the primary evidence.

Hints

HintFollow one document
Start with a document ID, read its inventory and measurements, then find its findings and selected example before writing a conclusion.
HintUse the exact boundary
“This file has 14 decoded code points” is precise. “This document is short and simple” adds an unsupported judgment.

Review

Read the report as a new reader. Can they tell which bytes were accepted, how lines were counted, why a document was flagged, where a figure value came from, and what remains unknown? Replace broad wording with an artifact-backed statement or remove it.

How to check your work

Checks compare the report structure and evidence links with the supplied fixture. The supplied fixture is an exemplar of traceable writing, not a conclusion to copy without checking the corpus.