Apply inspection rules and select examples

An inspection finding invites a closer look. It is not a verdict about the text. Keep findings, final states, and examples separate.

Goal

Apply the configured rules, produce traceable findings, select deterministic escaped excerpts, group duplicate content, and account for every document as accepted_clean, accepted_flagged, or rejected.

Inputs

Use the verified inventory, accepted and rejected decode records, line analysis, measurements, and plans/inspection_config.json. The configuration supplies thresholds and exact rules for:

• an empty document;
• a decoding failure;
• mixed line-boundary styles;
• a line longer than the configured code-point limit;
• a control code point outside allowed tab and line boundaries;
• leading or trailing Unicode whitespace in a line; and
• an original-byte digest shared by more than one document.

An accepted document may have several findings. A finding is one document/rule pair with measured evidence and a short configured explanation.

Deliverables

Implement rule evaluation in src/inspect.py and save:

• output/inspection_findings.csv;
• output/selected_examples.json; and
• output/duplicate_content_groups.json.

Give every listed document exactly one final state. Select examples for each applicable rule by document ID and line number, keep the configured number, escape control characters, preserve source location, and include an explicit truncation marker when the configured excerpt length is exceeded.

Do not copy shortened excerpts into the corpus. Do not turn a finding on a successfully decoded document into rejection unless the stated state rule says so; the accepted packet's default is accepted_flagged.

Checks

Use fixtures that create several findings for one document, a duplicate byte digest under different IDs, a disallowed control code point, a mixed boundary, and an overlong line. Check that each finding names document ID, rule, measured evidence, and explanation.

Check the three final-state counts sum to the complete manifest count, that examples are ordered deterministically, escaped, source-located, and bounded, and that duplicate groups preserve all document IDs. Verify no source bytes or analysis text are mutated.

Workspace

Keep rule evaluation and example selection in src/inspect.py. Read supplied thresholds from src/config.py and measurements and line records from earlier modules. Write findings and example artifacts under output/.

Hints

Review

Read one document with multiple findings and one clean document. Explain why the final state and each finding answer different questions, and trace one escaped excerpt back to its original line.

How to check your work

Checks compare finding names, final states, duplicate groups, and example ordering with the supplied fixture. The supplied fixture keeps evidence inspectable rather than converting it into a quality label.