Milestone 1 of 10

Verify source files, UTF-8, identities, configuration, and bounds

Check safe manifest paths, exact bytes and digests, strict UTF-8, format identities, dictionary limits, fixed width, and positive parser bounds.

The codec must know exactly which bytes it is allowed to read. Verify safe paths, digests, strict decoding, format rules, and positive limits before codec state exists.

Goal

Validate manifest-listed source files, byte counts, SHA-256 digests, strict UTF-8 text, source identities, codec versions, primary and audit limits, fixed width, and all configured size bounds.

Inputs

Read the versioned source manifest, repository-owned files, fixtures, and plans/codec_config.json. A manifest entry has a unique document ID, safe relative path, byte count, SHA-256 digest, and text identity. Read each file in binary mode and decode its complete bytes with strict UTF-8 before accepting it as text.

The configuration supplies primary and ordered audit dictionary limits, accepted container version, source and payload/code bounds, fixed code width, size-accounting fields, and plot order. Limits are positive non-boolean integers. Dictionary limits are unique integers from 256 through 65536, and the primary value appears once in the audit list. The code width is exactly two bytes.

Deliverables

Implement validation in src/config.py and source loading in src/sources.py. Produce output/source_inventory.csv with document ID, safe path, byte count, digest, strict-text status, text identity, and source identity.

Checks

Reject unsafe or unlisted paths, missing or duplicate IDs, changed byte counts, changed digests, changed text identity, invalid UTF-8, changed source identity, unsupported versions, non-two-byte code width, duplicate or out-of-range audit limits, missing primary setting, and non-positive or boolean-like bounds.

Check empty, ASCII, repeated, non-repeated, newline, punctuation, underscore, and multi-byte Unicode text. Confirm source bytes, decoded text, and manifest records remain unchanged.

Workspace

Keep source loading, digest checks, strict decoding, identity, and configuration checks in src/config.py and src/sources.py. Do not initialize dictionaries, encode codes, or create payloads yet.

Hints

HintBytes come first

Read bytes, count bytes, hash bytes, then decode strictly. Character count is not a substitute for the manifest byte count.

HintValidate the whole audit list

Check every limit and its order, then check that the primary value appears exactly once. Do not infer a default limit.

HintA safe path is part of identity

A digest cannot make an unsafe relative path acceptable. Validate the path boundary before opening the file.

Review

Follow one Unicode document from path through bytes, digest, and decoded text. Which value would remain unchanged if someone incorrectly measured characters instead of UTF-8 bytes?

How to check your work

Checks compare inventory rows, identities, strict-decoding outcomes, limits, and rejection reasons with the fixtures. The supplied fixture accepts only manifest-listed bytes that satisfy every boundary.