Milestone 9 of 11

Prove search-behaviour equivalence

Replay the same exact-term searches against both indexes and compare query analysis, scores, ranks, truncation, and excerpts field by field.

Two indexes can be structurally identical yet be used under different query rules. Replay the complete TEXT-04 search contract against both indexes.

Goal

Run every supplied query through the source and rebuilt indexes and compare analysis, scoring contributions, candidates, ranks, returned IDs, and excerpts field by field.

Inputs

Use the source and rebuilt indexes, the same corpus and tokenizer identities, queries, scoring and tie configuration, top_k, excerpt configuration, and expected search artifacts. Encoding must not alter query analysis, ignored or missing terms, contribution rows, candidate counts before truncation, scores, rank order, returned result IDs, or excerpt token positions.

Deliverables

Implement search comparison in src/equivalence.py. Produce source-versus- rebuilt comparison records for query analysis, contributions, pre-truncation candidate counts, scores, rank order, returned IDs, and excerpt token positions. Each record names query identity, field path, expected value, observed value, and agreement or first mismatch.

Checks

Replay all TEXT-04 query cases, including empty, repeated, shared, case-different, non-ASCII, underscore, ignored, and missing terms. Check all ranking ties, result limits, score fields, candidate counts before truncation, returned IDs, and source token positions in excerpts.

Deliberately change a rebuilt posting or query configuration in a separate fixture and verify the first behavioral mismatch is reported. Check structural and behavioral comparisons remain separate, deterministic, and input-immutable.

Workspace

Keep search replay and field comparison in src/equivalence.py. Read source and rebuilt artifacts without mutation. Write source-versus-rebuilt evidence; do not calculate size accounting or the final manifest yet.

Hints

HintCompare before truncation

A matching returned top-k list can hide a changed candidate set. Compare the complete candidate count before applying the result limit.

HintKeep query evidence

Compare ignored and missing terms as well as scores. A result match under different analysis rules is not behavioral equivalence.

HintUse stable paths

Report a mismatch such as queries[q1].results[2].document_id, not only “search differs.”

Review

Trace one query from token analysis through contributions, rank, result ID, and excerpt position in both runs. Which field would reveal a changed tie rule even if the top result stayed the same?

How to check your work

Checks compare field-level search records with the expected fixtures. The supplied fixture requires complete agreement under the same TEXT-04 rules and reports the first difference rather than treating similar results as equal.