LLM Primer

Introduction

Probability is the mathematics of uncertainty.

In machine learning, uncertainty appears in several central places:

  • data is sampled from a larger world
  • labels may be noisy
  • model outputs are often probabilities
  • training objectives compare predicted and observed outcomes
  • evaluation asks how often a model is right, wrong, calibrated, or uncertain

Probability gives us the language for these ideas.

sample spaceevent A
An event is a set of outcomes inside a sample space.

The Main Idea

Probability assigns numbers to uncertain outcomes.

A probability is between 0 and 1.

  • 0 means impossible
  • 1 means certain
  • values between them describe uncertainty

These numbers are not decorations. They must obey rules. If probabilities do not add up consistently, the model of uncertainty is broken.

For example, if a classifier says one image has probabilities:

cat: 0.7
dog: 0.2
car: 0.1

then it is describing uncertainty across three possible labels. The values sum to 1, so they can be read as a distribution over those labels.

Probability Is About Models

Probability does not remove uncertainty. It gives uncertainty a structure we can compute with.

When we say "the probability of rain is 0.3," we are not saying rain happens 30 percent today. We are saying that, under our model and available evidence, rain has probability 0.3.

This distinction matters in ML. A model output may look precise, but it is still a statement made by a model under assumptions.

What This Chapter Covers

This chapter introduces:

  • sample spaces and events
  • probability rules
  • random variables
  • distributions
  • expectation
  • variance and covariance
  • independence
  • conditional probability
  • Bayes' rule
  • common ML distributions

The goal is working fluency, not measure-theoretic probability.

What We Will Keep Simple

We will work with finite examples first: dice, labels, and small tables of probabilities. Later ideas can handle continuous distributions and more formal probability spaces.

The goal here is to learn the habits needed for ML: define the possible outcomes, assign probabilities consistently, and update them when new information arrives.

Before Moving On

You should be comfortable with sets, functions, sums, vectors, and basic calculus notation.