Milestone 6 of 8

Apply image groups and form batches

Select every aligned field through shared group indexes, prove disjoint complete coverage, keep smaller final batches, and represent zero-patch images and groups honestly.

Grouping and batching must move all aligned fields together. A smaller final batch is still part of the dataset.

Goal

Select image groups through shared indexes, prove conservation and disjointness, batch each group in saved patch order, and retain every remainder identity.

Group contract

image_groups.csv assigns every image exactly once to working, checking, or reserved. Select patches by their image's group. Group patch sets are disjoint and together cover the complete aligned patch set. An image with no patches remains in image summaries; a group with no patches creates zero batches.

Batch contract

Batch each group separately in saved patch order. Keep a smaller final batch when the count is not divisible by batch size. Record each batch's group, ordinal, size, and complete composite identities. Do not drop, duplicate, mix, or independently reorder patches, masks, and metadata.

Deliverables

Implement shared-index grouping in src/batches.py. Save per-group patch and mask archives, aligned metadata indexes, output/batch_manifest.json, and group summaries containing image and patch counts.

Checks

Check group identity conservation, disjoint patch indexes, complete union, aligned array selection, and saved order. Check a group with zero patches, groups with one patch, and a remainder smaller than the configured batch size. Follow the final remainder's composite identities into its patch, mask, and metadata rows.

Deliberately duplicate an image assignment, omit one, mix a group index, or drop the final remainder. The verifier must report the first disagreement. Confirm source, aligned arrays, and group assignments remain unchanged.

Workspace

Keep grouping and batching in src/batches.py. Do not add random sampling, epochs, reshuffling, or training terminology.

Hints

HintSelect by image identity
A patch's row number is not its group. Use the image ID carried by metadata and apply one shared index.
HintName the small batch
The final batch has fewer rows because the group count is not divisible by the configured size. Record its identities; never hide it.

Review

Pick one patch in the final batch and find its matching mask and metadata row. What would a group overlap or missing remainder do to the dataset count?

How to check your work

Checks compare group counts, indexes, batch sizes, order, and identities with the fixtures. The project makes one deterministic pass; it does not define a sampling strategy.