Milestone 7 of 9

Inspect bounded group-size and threshold sensitivity

Rebuild each accepted setting and separate eligibility changes from threshold-only additions and removals without choosing a best parameter.

Sensitivity changes the rule in two different ways. Rebuild each audit setting and keep eligibility effects separate from threshold effects.

Goal

Repeat the complete construction for every ordered audit group size and rational threshold, then report counts and exact flagged-pair additions and removals without choosing a best setting.

Inputs

Use validated documents and configuration audit settings. For every setting, rebuild selected groups, unique sets, occurrences, eligibility, pair metrics, and threshold decisions. The primary setting remains the published result.

Changing group size changes eligibility and group identity. Changing threshold changes selection but not pair similarity at a fixed group size. For every setting save eligible-document count, eligible and excluded pair counts, flagged-pair count, and exact flagged pair IDs. Relative to primary, save pair additions and removals.

Deliverables

Implement src/sensitivity.py. Produce output/sensitivity_summary.csv with setting identity, group size, threshold numerator and denominator, counts, primary marker, and ordered exact flagged IDs. Produce additions and removals for every audit setting, with exact plot data for sensitivity figures.

Checks

Check one and several group sizes, thresholds zero and one, duplicate-setting rejection, primary-setting inclusion, short documents, repeated groups, and changes that affect eligibility versus only selection. Verify each setting is rebuilt rather than copied, pair counts conserve the complete quadratic table, and additions/removals are set differences against primary flagged IDs.

Check that audit rows do not replace the primary result, no setting is called best, and changing threshold at fixed group size leaves pair similarity and eligibility unchanged. Confirm deterministic setting and pair order.

Workspace

Keep audit rebuilding, comparison, and sensitivity summaries in src/sensitivity.py. Read primary artifacts without mutation. Write summaries, exact pair additions/removals, and plot data; do not serialize the full project yet.

Hints

HintChange one cause at a time

Compare equal group sizes with different thresholds to isolate selection. Compare equal thresholds with different group sizes to expose eligibility and group identity changes.

HintUse exact IDs

Store pair IDs in deterministic order, then compute additions and removals as set differences. Do not compare only counts.

HintKeep primary visible

Mark the primary row and keep its flagged list unchanged. The audit explains sensitivity; it does not select a winner.

Review

Find two audit rows with the same group size and different thresholds. What may change, and what must not? Then compare two group sizes at one threshold.

How to check your work

Checks compare sensitivity counts, exact IDs, additions, removals, and primary markers with the fixtures. The supplied fixture reports effects on this corpus without tuning or selecting a preferred setting.