Form one deterministic pass of batches

A batch is an ordered slice of one group. A small final batch is evidence, not waste.

Goal

Form one deterministic pass of batches for each group, keep every remainder, and preserve alignment and composite identity inside each batch.

Inputs

Use each group's saved ordered arrays and the positive batch_size from sequence_config.json. Batch groups separately in their saved example order. The fixed policy is keep: a final batch may contain fewer than batch_size rows.

Deliverables

Implement batching in src/batches.py. Save batch records and output/batch_manifest.json with group, batch index, exact row count, and the composite identity of every row. Record full-batch and remainder counts.

Checks

Test batch sizes below, equal to, and above the group count. Check that every full batch has exactly batch_size rows, the final smaller batch is retained, and no row is dropped, duplicated, wrapped, or borrowed from another group. A group with zero examples produces zero batches, not an invented empty batch.

Within every batch check context shape (rows, C), mask shape (rows, C), target shape (rows,), metadata length, mask/pad invariants, and identity alignment. Check concatenating batches in saved order recovers the group's arrays exactly. Do not introduce epochs or automatic reshuffling.

Workspace

Keep batching in src/batches.py. Treat the source group arrays as read-only and write batch evidence under output/.

Hints

Review

Choose a group whose count is not divisible by the batch size. Identify its last batch and show that its rows appear once in the group and once in the manifest.

How to check your work

Checks compare counts, ordering, remainder identities, and aligned shapes with the supplied fixture. The supplied fixture implements one pass only.