Milestone 7 of 11

Wrap and validate the binary container

Add the LMP magic, accepted version, and big-endian payload length, then reject malformed outer containers in a defined order.

The outer container gives the payload a version and a declared boundary. Apply that small format exactly before asking the payload parser to interpret fields.

Goal

Write and validate the FILE-15 outer container with exact magic, version, and big-endian payload length, including every required truncation and trailing-byte failure.

Inputs

Use the complete payload, accepted container version, configured payload-byte limit, and format identity. The layout is:

bytes 0..2   magic b"LMP"
byte 3       container version, from 0 through 255
bytes 4..7   payload length as an unsigned four-byte big-endian integer
remaining    exact payload bytes

Writing rejects an unsupported version or payload length above the configured limit. Reading checks, in order, truncated header, magic, accepted version, declared payload length, truncated payload, trailing container bytes, and payload bounds before parsing index fields.

Deliverables

Implement src/container.py and write canonical output/index.bin. Save its SHA-256 digest and container metadata, including magic, version, declared and actual payload length, and source identity.

Checks

Check the exact header bytes, unsigned big-endian length, accepted version, payload boundary, deterministic bytes, and configured payload limit. Exercise a truncated header, changed magic, unsupported version, truncated payload, excessive declared length, trailing container bytes, and a payload that would only appear valid if the container boundary were ignored.

Check malformed containers fail in the published order before payload field parsing, valid payload bytes are passed unchanged, and input payload and source records are not mutated.

Workspace

Keep header writing and boundary validation in src/container.py. Write index.bin, digest, and rejection evidence. Do not fold container failures into payload-structure failures or run equivalence yet.

Hints

HintSeparate the two boundaries
The header bounds the payload; the payload parser bounds its fields. Validate the first before interpreting the second.
HintRead length explicitly
Use unsigned four-byte big-endian decoding for bytes 4 through 7. Do not use host byte order or infer length from the remaining file.
HintTrailing bytes are not padding
After consuming exactly the declared payload, any container bytes left over are a rejection case.

Review

Take a valid payload and append one byte to the container. Which check should reject it, and why should the payload parser never see that byte?

How to check your work

Checks compare index.bin, digest, header metadata, and failure order with the container fixtures. The supplied fixture accepts one versioned layout and never guesses a format from content.

LLM PrimerWrap and validate the binary containerhttps://llmprimer.com/python/projects/compress-and-rebuild-an-inverted-index/wrap-and-validate-the-binary-container© 2026 LLM Primer