Milestone 1 of 8

Validate and inspect reference and query records

Check schemas, identities, coordinates, labels, manifest evidence, and k before calculating any distance.

Begin with the two supplied tables. Neighbor ordering is only meaningful when the rows and identities have a clear contract.

Goal

Validate the reference and query records, confirm their manifest identity, inspect the two-coordinate values, and accept a valid k before calculating any neighbor.

Inputs

Use the read-only files data/reference_records.csv, data/query_records.csv, and data/dataset_manifest.json. The reference file requires a unique non-empty record_id, finite value_1 and value_2, and a non-empty label. The query file requires a unique query_id, finite coordinates, and an expected_label for the bounded checking summary.

Use a configuration record containing integer k, with 1 <= k <= reference_count. Reference and query IDs are separate namespaces; do not compare an ID across the two files as if it identified the same record.

Deliverables

Implement src/data.py and src/config.py to load and validate the tables, manifest, coordinate width, identities, labels, and k. Preserve source row position for every reference record and query record.

Record the dataset identity, schema, row counts, coordinate width, and one small inspection of the supplied points in the report. A simple plot of the two-coordinate records may be created, but do not interpret visible groups as causes or clusters.

Checks

Check the manifest identity and headers, unique IDs, equal coordinate width, finite coordinate values, non-empty labels, and valid k. Reject empty tables, duplicate IDs, missing fields, non-finite values, wrong widths, and a k outside the reference count before distance calculation begins.

Check that source order and IDs are preserved and that loading does not mutate the supplied files or arrays. Use public fixtures for one malformed reference row, one malformed query row, and an invalid configuration.

Workspace

Keep file and manifest loading in src/data.py; keep k validation in src/config.py. Leave all supplied files read-only. Do not create neighbor or prediction artifacts until the next milestones.

Hints

HintIdentity is part of the row

Keep the original row position beside every ID. Sorting later must not erase the position needed to break an equal-distance tie.

HintValidate the boundary before the values

A file with plausible coordinates can still have the wrong schema or dataset identity. Compare headers and manifest fields first.

Review

Read the validation result against the actual CSV headers and rows. Explain why an expected label is only a checking field in this project and why a visible pattern in the coordinate plot is not an explanation.

How to check your work

Checks compare the loader, manifest checks, and invalid-input messages with the supplied fixture. The supplied fixture preserves the supplied schema; it does not invent a replacement.