Construct examples for one tiny document

Before building a table, work out one document by hand. The target is the next ID, and its context contains only earlier IDs.

Goal

Construct every context–target pair for one tiny encoded document and preserve its document ID, target position, context start, and real-context length.

Inputs

Use one valid sequence such as <bos> a b <eos> and a configured context width C. For every target position t from 1 to len(ids) - 1, take ids[max(0, t - C):t]. Do not take a target from one document while looking back into another.

The first valid target is at position 1, so <bos> can be its only real context. The final target is <eos>. <bos> supplies context but is never a target. If an earlier position happens to contain the same ID as the target, that earlier occurrence is still legitimate context; position, not equality, defines the boundary.

Deliverables

Implement the unpadded example builder in src/examples.py. Save a hand-check fixture with one row for each target position and fields for document ID, target position, context IDs, context start, and real-context length. Keep the source sequence unchanged.

Checks

Use a valid document with one ordinary source token, another with several tokens and repeated IDs, and an empty document. The one-token source encodes as <bos> token <eos> and therefore creates two examples. Check that every valid non-empty encoded sequence creates exactly len(ids) - 1 examples, while an empty encoded sequence creates zero. Reject a one-ID encoded sequence because it cannot contain the required boundary pair. Verify target positions are increasing, context ends immediately before the target, and contexts never cross a document boundary.

Check widths shorter than, equal to, and longer than available history. Prove the composite identity (document_id, target_position) is unique and that the source sequence is not mutated.

Workspace

Keep preceding-context selection in src/examples.py. Do not add padding or group-specific reordering yet; those belong to later milestones.

Hints

Review

Explain one row from the hand fixture without referring to a model. Which IDs were available at that target position, and why is the target absent from its own context?

How to check your work

Checks compare positions, contexts, counts, and identities with the version-matched fixture. The supplied fixture makes no prediction about what a model will learn.