Form working batches

The working group is still ordered evidence. Batching should divide that order without dropping a position silently.

Goal

Partition the working positions into ordered batches of at most batch_size, and make the remainder decision explicit.

Inputs

Use the working indexes from the shared split and the validated batch_size and keep_remainder values. A full batch contains exactly batch_size positions. If positions remain after the full batches, keep one smaller final batch when keep_remainder is true; otherwise omit those positions and record their IDs explicitly.

This is one pass through the current working order. Do not add epochs, automatic reshuffling, prefetching, or framework data-loader behavior.

Deliverables

Implement a batch iterator or result record containing:

• ordered batch positions;
• ordered IDs for every batch;
• the batch size and remainder policy;
• IDs omitted when a remainder is not kept;
• counts that account for every working position.

The first batch must begin with the first working position. Do not sort by ID or by a numerical field.

Checks

Test a working group shorter than one batch, an exact division, one leftover position, several leftover positions, and a batch size larger than the working group. Run both remainder policies.

For keep_remainder=True, every working position must occur in exactly one batch. For False, every position must occur in one batch or in the explicit omitted list, never in neither and never in both. Check that each batch has no more than the requested size and that source IDs remain attached.

Workspace

Implement the batch rule in src/batches.py. Keep omitted IDs as evidence in the returned result so main.py can save them in the batch manifest.

Hints

Review

Compare the batch IDs with the working IDs in their order. Explain why an omitted ID is different from a record that was never present in the source.

How to check your work

The supplied fixture handles exact, short, and oversized batch boundaries and records omitted IDs when the remainder is not retained. It does not introduce an epoch or a data-loader abstraction.