Learn Machine Learning and Large Language Models from First Principles

The curriculum begins with Python and mathematics, then continues through neural networks, language models, and transformers. Lessons include worked calculations, code, exercises, and interactive diagrams.

Start hereView curriculum

Understanding large language models

A large language model processes text as tokens and estimates which tokens are likely to come next. Modern models learn these estimates from large text collections using neural networks built around the transformer architecture.

Understanding the complete system requires programming, vectors and matrices, probability, optimization, neural networks, language modeling, and attention. The curriculum introduces these subjects in that order.

Inside a language model
  1. Text
  2. Tokens
  3. Embeddings
  4. Transformer layers
  5. Token probabilities
Curriculum path
  1. Python
  2. Mathematics
  3. Deep learning
  4. Language modeling
  5. Transformers
  6. LLM systems

Lesson structure

A lesson usually begins with a concrete problem. It then introduces the relevant idea, works through a small example, and asks you to use it yourself.

Diagrams are interactive when changing a value helps explain the result. Runnable examples show the arrays, weights, activations, and errors produced by the code.

  1. 1

    Problem

    Begin with the question the method must answer.

  2. 2

    Explanation

    Introduce the idea and its mathematical form.

  3. 3

    Worked example

    Calculate a small case by hand.

  4. 4

    Code

    Implement the same calculation in readable code.

  5. 5

    Practice

    Solve exercises and test the method on new cases.

Software and tools

Early lessons use Python, NumPy, and Matplotlib so that the arrays and calculations remain easy to inspect.

Later lessons introduce PyTorch and other frameworks for larger models, GPU computation, and practical training workflows.

Who the course is for

The first subjects assume no previous study of machine learning. Readers with programming or mathematics experience can begin at the first unfamiliar chapter.

The course is intended for readers who want to understand the calculations, implement the methods, and test what they have learned.

Curriculum

Subjects are arranged in prerequisite order. Python and mathematics come before deep learning; deep learning comes before language modeling and transformers.

Python

Write and debug small programs, then use NumPy and Matplotlib for numerical work.

Math

Learn the vectors, matrices, probability, calculus, and optimization used in later subjects.

Deep Learning

Build neural networks from tensor operations, loss functions, gradients, and training loops.

Language Modeling

Represent text with tokens and study how models assign probabilities to sequences.

Transformers

Develop attention and transformer blocks from the operations introduced earlier.

Generative Models

Study autoencoders, diffusion, flow matching, and other approaches to generation.

RL Foundations

Work from states, actions, and rewards to value functions and policy gradients.

Advanced subjects

Later subjects cover reinforcement learning for language models, world models, and the systems used to evaluate and deploy modern models. Research papers are cited where they introduce the methods being studied.

RL for LLMs

Study reward models, preference learning, RLHF, RLAIF, and policy optimization.

World Models

Learn how models represent environments and predict the consequences of actions.

Modern LLM Systems

Study retrieval, tool use, evaluation, agents, and deployment.

Projects and exercises

Exercises and projects include numerical plots, gradient checks, training loops, tokenizers, attention blocks, and controlled experiments. Each one uses material developed in the preceding lessons.