Milestone 2 of 10

Map text bytes and hand-trace the initial dictionary

Separate characters from UTF-8 byte values and establish the exact 256-entry literal dictionary for empty, ASCII, repeated, and non-ASCII inputs.

The codec operates on byte symbols, not Python characters. Establish the literal dictionary and trace its state on small inputs before growing phrases.

Goal

Map source text to its exact UTF-8 bytes, establish the 256 literal dictionary entries, and hand-trace empty, ASCII, repeated, and non-ASCII examples.

Inputs

Use validated source records and the fixed initial dictionary:

(0,)   -> code 0
(1,)   -> code 1
...
(255,) -> code 255

Dictionary phrases are non-empty tuples of byte integers. New codes begin at 256. Dictionary state resets for every document; an empty source emits no codes and ends with dictionary size 256.

Deliverables

Implement byte mapping and initial state in src/dictionary.py. Produce hand-check records showing document ID, source text, UTF-8 bytes, initial entries, phrase lookups, and final dictionary size for empty, ASCII, repeated, and non-ASCII examples.

Checks

Check literal byte values 0, 1, 127, 128, and 255; UTF-8 multi-byte text; repeated bytes; newline and punctuation; empty input; and separate documents with separate dictionary state. Verify each initial phrase is one byte, codes are 0..255, new code allocation starts at 256, and no Python character becomes one dictionary symbol unless its UTF-8 encoding has one byte.

Check dictionary phrases are non-empty byte tuples and source records remain immutable.

Workspace

Keep byte conversion and initial dictionary helpers in src/dictionary.py. Write hand-trace evidence only. Do not implement phrase growth or decoding yet.

Hints

HintPrint the bytes

Use text.encode("utf-8") and inspect the integer values. A character such as é contributes more than one byte.

HintReset at the document boundary

Construct a fresh literal map for every source file. A phrase learned in one document cannot enter another.

HintKeep the tuple

Use (byte,) and longer tuples as dictionary keys. A joined display string would confuse bytes and text.

Review

Compare the UTF-8 bytes for one ASCII character and one multi-byte character. Why must the dictionary operate on the byte sequence rather than the visible text characters?

How to check your work

Checks compare byte mappings, literal entries, empty behavior, and final size with the hand-worked fixtures. The supplied fixture starts every document with exactly 256 literal byte phrases.