Milestone 4 of 8

Select groups through shared indexes

Apply each index array to every field and verify coverage, uniqueness, disjointness, and row alignment.

An index array is useful only if every field follows it. This milestone turns the two position groups into records while keeping their identity visible.

Goal

Select record_id, sensor, observation, and reading through the same checking and working indexes, then prove that the resulting groups preserve source identity and coverage.

Inputs

Use the four source arrays and the checking and working index arrays from the previous milestone. Every source position belongs to exactly one group.

The selection operation may return arrays or records, but each output row must retain the original ID and its three associated values. The output order is the order of the supplied index array, not a newly sorted order.

Deliverables

Implement an aligned selection function and an audit result containing:

  • ordered checking and working IDs;
  • each selected row's sensor, observation, and reading;
  • source positions used for each row;
  • coverage, disjointness, and uniqueness evidence.

Do not independently sort, filter, or shuffle one column. Do not rename the fields as features or targets.

Checks

Use hand-checkable records whose IDs are easy to distinguish. Check that every source position appears in one group, no position appears twice, the groups are disjoint, and the combined order matches the permutation slices.

For several positions, compare the selected row with the original source row by ID and by position. Deliberately test a broken implementation that shuffles one field separately; equal shapes must not make that implementation pass. Verify that all four source arrays remain unchanged.

Workspace

Implement selection and invariant reporting in src/aligned_data.py or src/split.py, keeping the boundary clear. The later batch stage should consume the working positions rather than reconstructing them from IDs.

Hints

HintCheck IDs and values together
A matching position is evidence only when the ID and every field at that position still match the source.
HintCoverage has three parts
Check that the groups cover every source position, do not overlap, and contain no repeated position.

Review

Trace two rows from the source into different groups. Explain why a table with the right number of rows can still be wrong if one column was selected with a different index order.

How to check your work

The supplied fixture selects all four columns through the same index arrays and returns explicit invariant evidence. Compare the evidence fields with the source rather than accepting a shape-only check.

LLM PrimerSelect groups through shared indexeshttps://llmprimer.com/python/projects/build-a-reproducible-split-and-batch-pipeline/select-groups-through-shared-indexes© 2026 LLM Primer