Machine learning from first principles

Learn how large language models work, starting with Python and mathematics

LLM Primer teaches the programming, mathematics, and neural-network ideas behind language models. Lessons combine worked calculations with editable code, interactive diagrams, and exercises.

Start with PythonView curriculum

Free to read. No account required. The Python course assumes no previous programming experience.

Follow one predictionSmall transformer · 4 dimensions
Try:
Tokens
Themodellearnsfromdata
  1. 1Token vectors
  2. 2Positions
  3. 3Attention
  4. 4Probabilities

This small teaching model uses real transformer operations with simplified values. It is not a production LLM.

Inside the model

A transformer is a sequence of ordinary operations

A decoder-only transformer turns token IDs into vectors, uses attention to combine context, transforms each position, and converts the result into probabilities for the next token. The same block is repeated many times.

Explore the calculation step by step →
Decoder-only transformer architecturePrompt text is tokenized, converted to token and position vectors, passed through repeated pre-normalization causal-attention and feed-forward sublayers with residual paths, and projected into next-token probabilities.Prompt textAttention connects words to contextTokenizertoken strings → vocabulary IDsToken embeddingID selects one learned rowPosition informationencodes token order+Decoder-only transformer block× NInput vector XLayer normalizationCausal self-attentionQ, K, VprojectionsQKᵀ/√dscoresMaskcausal + softmaxWeights × Vcontexteach token sees only itself and earlier tokens+residualLayer normalizationFeed-forward networkup projection → activation → down projectionthe same learned transformation is applied at every position+residualFinal normalization + vocabulary projectionhidden vector → logitsSoftmax → next-token probabilities
A modern decoder-only language model. The original 2017 Transformer also included an encoder and cross-attention for sequence-to-sequence tasks. Normalization order and position methods vary across models.

How lessons work

Read the explanation, then try it yourself

Each new idea is paired with a small calculation, program, diagram, or exercise. Change an input, predict the result, and then check what happens.

01

Work through the numbers

Start with examples small enough to calculate by hand before moving to larger arrays and models.

[2, −1]+[1, 3]=[3, 2]
02

Change the code

Edit and run Python in the browser. Compare the result with what you expected.

values = [2, 5, 8]
mean = sum(values) / len(values)
print(mean)  # 5.0
03

Check your reasoning

Exercises ask you to predict results, perform calculations, repair programs, and explain your answer.

If every value in [2, 5, 8] doubles, what happens to the mean?Predict first → calculate → check

Curriculum

Python first, then mathematics and models

Subjects are arranged in prerequisite order. Begin with the first subject you do not already know. The labels show which subjects are open, under review, or planned.

View all subjects →
  1. 01
    PythonOpen
  2. 02
    MathematicsIn review
  3. 03
    Deep learningIn review
  4. 04
    Language modelingIn review
  5. 05
    TransformersPlanned
  6. 06
    LLM systemsPlanned

Begin at the first unfamiliar idea

If programming is new, begin with Python. If you already write Python comfortably, use the curriculum to find the first unfamiliar chapter.

Start with Python

Begin with how programs work, then learn functions, data structures, NumPy arrays, plotting, debugging, and small numerical experiments.

Open PythonRead the learning guide