Build deterministic results, excerpts, and figures

The matching functions can find evidence in different orders. This milestone turns that evidence into stable results and source-linked views without hiding the positions that support them.

Goal

Apply the published result order, build one exact excerpt per match, and save figures whose plot data can be inspected and regenerated.

Inputs

Use analyzed queries, phrase and proximity matches, traces, complete scanner tokens, source documents, and the configured non-negative excerpt radius.

Across queries, preserve query-file order. Within a query, order by corpus-manifest document order, increasing source span, and role positions as the final exact tie. Return every match; this is evidence and display order, not relevance ranking.

For each match, start at the smallest through largest matched full-token position. Expand by at most the configured number of complete scanner tokens on each side and clip at the document boundary. Do not merge overlapping excerpts.

Deliverables

Implement src/excerpts.py and src/report.py. Produce ordered phrase and proximity result records, one result_excerpts.jsonl record per match, and query_summary.csv. Each excerpt stores its half-open full-token range, exact token records, matched positions, and prefix/suffix truncation flags. Joined text is display data only.

Save candidate-versus-match and proximity-gap figures plus exact output/plot_data.json. Keep candidate counts, match counts, gap values, and query IDs in the plot data so the figures can be recreated without reading pixels.

Checks

Check query order, manifest order, source-span order, and role-position ties. Check source-edge excerpts, overlapping excerpts, zero-radius excerpts, punctuation and whitespace, and matches whose full-token positions are not adjacent. Verify that every excerpt token is copied exactly from the source, that its range contains the matched positions, and that truncation flags agree with clipping.

Check plot arrays against the saved match and trace records, deterministic labels and ordering, and no fabricated zero-match filler. Do not inject markup, alter corpus tokens, merge excerpts, add relevance scores, or present a polished search-results mockup as evidence.

Workspace

Keep result ordering, excerpt construction, and plot-data generation in their separate modules. Read prior artifacts as inputs and write ordered results, excerpts, summaries, figures, and plot_data.json under output/.

Hints

Review

Choose two matches from different query kinds and explain why their order is stable. Inspect an excerpt at each source edge. Can someone reconstruct the plotted candidate and gap values from the JSON data alone?

How to check your work

Checks compare ordered results, excerpt ranges, flags, and plot data with the tiny fixtures. The supplied fixture preserves source evidence and exact ordering; it does not add ranking or silently collapse overlapping views.