Code Points, Bytes, and Graphemes

A byte is a storage unit. A Unicode code point is a numbered character in the Unicode standard. A grapheme cluster is what a reader often experiences as one visible character.

These can differ.

For example, an accented character may be stored as one precomposed code point or as a base letter plus a combining mark. A flag emoji may look like one symbol but involve multiple code points.

The lesson for language modeling is practical: counting "characters" is not always the same as counting bytes, code points, or visible symbols.

LM-C03-T01-001Exercise: Bytes versus visible text

Enter 1 if byte length and visible character count are always the same, or 0 if they can differ.

Compute it first, then check your number.

Later tokenizers may operate on bytes, characters, subwords, or text after normalization. Know which layer you are using.