LLM Primer

A living introduction to language models

How do we build intelligence?

Language models from first principles.

Through mathematics, code, and experiment.

Illustrative neural signalling; connection changes are accelerated.

Drag to orbit · right-drag or two fingers to pan · scroll or pinch to zoom

Touch a neuron. Follow the signal.

networksignalguidancegrowingfading
Loading field

The journey

From signals to language models.

The illustration begins with biological neurons, but the book studies artificial systems. We start with text and prediction, build neural networks and Transformers from their smallest parts, then follow the path through pretraining, adaptation, reinforcement learning, inference-time computation, agents, and bounded self-improvement loops.

Learning

The chapters we plan to build.

Lessons
Learn each idea through explanation, worked examples, and short exercises.

Problems
Work through a precise question with the tools at hand.

Projects
Build, measure, and investigate a larger working model.

  1. 01IntelligenceMake the question concrete.
  2. 02Text DataShape raw text into trustworthy examples.
  3. 03Count ModelsPredict the next token from counts.
  4. 04Likelihood and LossMeasure how well predictions fit.
  5. 05Tensor ComputationExpress small computations with tensors.
  6. 06Neural NetworksBuild a predictor from learned layers.
  7. 07DifferentiationTrace how loss changes each parameter.
  8. 08OptimizationUse gradients to improve the model.
  9. 09Learned RepresentationsTest what the network has learned.
  10. 10Automatic DifferentiationLet software carry the gradients.
  11. 11Sequence ModelsCarry information through a sequence.
  12. 12AttentionLet each position choose what matters.
  13. 13The TransformerAssemble and train a complete decoder.
  14. 14Modern DecodersRefine the decoder for modern practice.
  15. 15PretrainingScale prediction across data and compute.
  16. 16Model AdaptationChange a base model for a purpose.
  17. 17Policy GradientsLearn actions from sampled rewards.
  18. 18Preference ModelsTurn comparisons into a learned signal.
  19. 19Preference OptimizationUse preferences to update behavior.
  20. 20Verifiable RewardsTrain against outcomes we can check.
  21. 21Inference-Time ComputationSpend computation to improve an answer after training ends.
  22. 22AgentsConnect models to actions and consequences.
  23. 23Self-Improving SystemsBuild and audit bounded improvement loops.