Shuffling and Splitting Paired Data

Shuffle example indexes rather than independently shuffling related arrays. Apply one permutation to every aligned array, divide the ordered indexes into explicit partitions, and verify that each feature row remains paired with its target.

Many datasets store related information in separate arrays. One array may hold the input features and another the target values:

Position i has one meaning across all three arrays. If feature[2] belongs to example 103, then target[2] must remain the target for that same example. Shuffling or splitting the dataset must preserve this positional alignment.

Represent a reordering with indexes

A permutation is a rearrangement of a complete set of indexes. For six examples:

One possible permutation is:

[4, 0, 5, 2, 1, 3]

Every original index from 0 through 5 appears exactly once. The permutation describes a new order without changing the original arrays.

Apply it to every aligned array:

The values at each new position still come from one original row.

This pattern is safer than independently shuffling each array:

Those calls generally use different permutations. The arrays retain equal lengths, but their semantic pairing is broken.

Distinguish permutation from in-place shuffle

NumPy provides two related operations:

permutation = rng.permutation(index)

permutation returns a reordered copy and leaves index unchanged.

rng.shuffle(index)

shuffle changes index in place and returns None.

Both can produce a valid index order. Returning a new permutation is often clearer in instructional and experimental code because the original indexes remain available for comparison.

Whichever operation you choose, generate one index order and reuse it for all aligned data.

Split one permutation at explicit boundaries

Suppose six examples should be divided into four training examples and two test examples:

Then select every aligned array with the corresponding indexes:

The two index arrays are disjoint: no position should appear in both. Together, they contain every example exactly once.

For a fractional split, calculate an integer boundary explicitly:

With ten examples, int(10 * 0.75) is 7, so the split contains seven training examples and three test examples. State this rounding rule rather than assuming every fraction produces an exact count.

Give each split one role

A common supervised-learning workflow uses:

• a training set to fit model parameters;
• a validation set to compare choices such as settings or model variants;
• a test set for a final evaluation after those choices are settled.

Repeatedly checking the test result while changing the program allows test information to influence development. The test set is then no longer an independent final check.

This chapter does not yet train a model, but the separation rule can be stated now:

fit on training data
choose with validation data
report once on test data

For a small learning example, a train/test split may be sufficient. When model choices depend on held-out results, introduce a validation split rather than using the test set for both selection and final reporting.

Verify the split numerically

The following demonstration checks alignment, coverage, and overlap. Change the seed or split boundary and rerun it.

Equal feature and target shapes do not prove alignment. The explicit target_by_id check verifies semantic pairing against stable example identifiers.

Keep preprocessing inside the training boundary

Splitting first is not enough if information from the test set enters an earlier computation. For example, computing a mean from the complete dataset and then using it to center both splits allows test values to influence the training transformation.

The safer order is:

split indexes
-> compute preprocessing settings from training data
-> apply those settings to training and held-out data

This problem is called data leakage: information that should be unavailable during fitting influences the fitted process.

A held-out input can still be transformed. The important distinction is that its values must not determine settings that are learned from the data.

Know when an ordinary random split is unsuitable

Randomly permuting individual rows assumes that rows may be exchanged without breaking the evaluation question. That assumption is not always valid.

Examples include:

• time-ordered data: training on future records and testing on earlier records may not represent a real forecast;
• grouped records: observations from one person, device, document, or location may need to stay in one split;
• rare classes: a small random split may contain too few examples of an important class;
• duplicate or related rows: near-identical records in different splits can make evaluation appear stronger than it is.

These cases require a split rule based on time, groups, class proportions, or another property of the data. A seed makes a chosen split repeatable; it does not make the split appropriate.

train_count = int(11 * 0.8)

train size: 4
test size: 2
overlap: 0
covered: 6
pairs valid: True

Retain the split contract

For each data split, record and verify:

unit being split
+ shared index rule
+ seed
+ split sizes and boundaries
+ role of each split
+ overlap and coverage checks
+ preprocessing boundary
= split contract

Then ask whether row-wise randomization matches the data's time, group, class, and duplication structure. Reproducibility preserves a split; experimental reasoning determines whether that split is valid.