Project 17

Build an Image Patch Dataset

Divide verified image arrays into fixed-size patches while retaining source coordinates, valid pixels, supplied groups, batch remainders, coverage, and reconstruction evidence.

  • 8 milestones
  • Optional
  • Browser workspace

Project question: Can you divide verified images into patches while keeping every source coordinate, valid pixel, mask, group, and batch aligned?

What makes this a project

A patch is more than a small array. It also has an origin, source bounds, padding decisions, a validity mask, and a place in a dataset. In this project you will generate a precise two-dimensional grid, extract patches without losing their spatial identity, measure what the grid covers, and save aligned arrays that another program can inspect.

This is a trustworthy spatial dataset boundary. It does not teach patch embeddings, attention, convolution, or model training. A small image whose pixels can be calculated by hand is the right test for a grid rule.

You will produce:

  • a checked image inventory, group assignment, and patch configuration;
  • exact row-major grids under drop and pad edge policies;
  • patches, validity masks, source coordinates, and composite identities;
  • aligned arrays and per-image coverage records;
  • exact reconstruction where coverage is complete and named gaps where it is not;
  • group-specific arrays and batches that keep their final remainder;
  • saved artifacts, figures, manifests, and a separate replay; and
  • a concise patch-dataset card with bounded claims.

What you should know first

The project follows the Python path through Chapter 13. You should be able to validate array shape, dtype, range, and source identity; reason about half-open slices and strides; pad arrays explicitly; keep masks and metadata aligned; accumulate in a safe dtype; batch shared indexes; visualize arrays; save NumPy artifacts; and test invariants. IMG-01 and DATA-02 are useful but optional. The supplied image and group records provide the boundary if you skipped them.

Supplied inputs

Use versioned grayscale or RGB arrays following IMG-01's (height, width, channels) uint8 contract. Each has an image_id, source and transformation identity, shape, and digest. All images in one run have the same configured channel count, either 1 or 3. Use plans/image_groups.csv, plans/patch_config.json, and the tiny hand-check arrays.

The group names working, checking, and reserved are operational labels for this project. Every image is assigned once, and all its patches stay in that image's group. They are not a claim about training, validation, testing, leakage, or sampling.

The project workspace

project/
  README.md
  data/                              # supplied, read only
  plans/image_groups.csv             # supplied, read only
  plans/patch_config.json            # supplied starting config
  src/config.py                      # reader implementation
  src/images.py                      # reader implementation
  src/grid.py                        # reader implementation
  src/patches.py                     # reader implementation
  src/coverage.py                    # reader implementation
  src/batches.py                     # reader implementation
  src/report.py                      # reader implementation
  src/main.py                        # reader implementation
  output/                            # generated artifacts
  tests/public_cases.py              # supplied, read only

Keep grid construction, extraction, coverage/reconstruction, grouping, batching, and reporting independently inspectable. Keep one workspace through every milestone.

Project milestones

Work through these in order. Each milestone produces evidence used by the next one, while project completion remains separate from lesson progress.

  1. 1Verify images, groups, and patch configuration

    Check source-array identities, one shared channel count, complete image groups, positive patch and stride dimensions, pad values, edge policy, batch size, and the fixed keep-remainder rule.

  2. 2Generate and hand-check patch grids

    Produce exact row and column origins under drop and pad policies, enumerate row first and column second, and test divisible, uneven, small, narrow, one-pixel, and overlapping cases.

  3. 3Extract patches with coordinates and masks

    Preserve image, origin, bounds, padding, indexes, group, patch values, and boolean validity masks while keeping each patch independent of its source.

  4. 4Build aligned arrays and metadata

    Stack patches and masks in global row-major order, keep metadata aligned through shared indexes, and reject duplicate spatial identities or mismatched rows.

  5. 5Measure coverage and reconstruct honestly

    Accumulate valid patch positions and values safely, prove source-times-coverage, reconstruct exactly when complete, and list uncovered coordinates when it is not.

  6. 6Apply image groups and form batches

    Select every aligned field through shared group indexes, prove disjoint complete coverage, keep smaller final batches, and represent zero-patch images and groups honestly.

  7. 7Inspect, save, and replay the dataset

    Save aligned arrays, tables, figures, coverage and reconstruction evidence, manifests, and batches; read them back; and reproduce them in a separate replay tree.

  8. 8Write and audit the patch-dataset card

    State source identity, axes, grids, edge and pad rules, spatial identities, groups, batches, coverage, reconstruction status, replay, and unsupported claims.

Required evidence

Save the versioned configuration, image inventory, and group identity; aligned patch and mask NumPy archives; patch_metadata.csv, image_patch_summary.csv, per-group archives, batch_manifest.json, coverage arrays and uncovered coordinates, exact reconstruction/equality records where defined, selected patch/mask/coverage/reconstruction figures with exact plot data, image_patch_dataset_manifest.json, a replay agreement or mismatch record, and report.md as the patch-dataset card.

Limits

This project does not teach patch embeddings, positional embeddings, attention, Transformers, convolution theory, random crops or sampling, resizing, interpolation, rotation, augmentation, labels, prediction targets, train/validation/test theory, leakage analysis, dynamic patch sizes, adaptive regions, object proposals, image pyramids, streaming, framework data loaders, GPU execution, compression, performance benchmarking, or claims that one patch policy is best for a model.

Review

The final review asks whether every patch has a unique spatial identity, grids are row-major and exact, padding and masks agree, arrays stay aligned, source times coverage is conserved, incomplete reconstruction is reported honestly, groups are disjoint and complete, remainders remain visible, replay is separate, and the card distinguishes observations from claims.