LLM Primer

Sample Spaces and Events

A sample space is the set of all possible outcomes.

If we roll a six-sided die, the sample space is:

Ω={1,2,3,4,5,6}\Omega = \{1, 2, 3, 4, 5, 6\}

An event is a subset of the sample space.

For example, the event "roll an even number" is:

E={2,4,6}E = \{2, 4, 6\}
sample spaceevent A
An event is a set of outcomes inside a sample space.

The sample space depends on what question we are asking. For a die roll, the outcomes may be numbers {1, 2, 3, 4, 5, 6}. For a classifier, the outcomes may be labels such as {cat, dog, car}.

An event is not necessarily one outcome. It can contain many outcomes. "Even number" contains 2, 4, and 6.

Probability of an Event

If all outcomes are equally likely:

P(E)=number of outcomes in Enumber of outcomes in ΩP(E) = \frac{\text{number of outcomes in E}}{\text{number of outcomes in }\Omega}

For the even-number event:

P(E)=36=0.5P(E) = \frac{3}{6} = 0.5

If outcomes are not equally likely, we do not count outcomes. We add their probabilities.

For a classifier distribution:

cat: 0.7
dog: 0.2
car: 0.1

the event "animal" is {cat, dog}. Its probability is:

0.7+0.2=0.90.7 + 0.2 = 0.9

That is a common source of mistakes. Counting outcomes works only when the outcomes have equal probability. A model distribution often assigns unequal probabilities, so event probability comes from adding probability mass.

ML Reading

A dataset is not the whole world. It is a sample from a larger space of possible examples.

Probability helps us reason about what we have seen and what we may see later.

This is why probability sits before statistics and machine learning evaluation. It gives us a precise way to talk about possible examples, possible labels, and events that combine many outcomes.

MATH-C07-T02-001Exercise: Compute event probability

A fair die has outcomes {1, 2, 3, 4, 5, 6}.

What is the probability of rolling a number greater than 4?

Compute it first, then check your number.

Hint

The favorable outcomes are 5 and 6.

Solution

There are 2 favorable outcomes out of 6, so the probability is 2/6 = 1/3, about 0.333. Counting works here because the die outcomes are equally likely.

MATH-C07-T02-002Exercise: Identify an event

For a die roll, the event "odd number" is {1, 3, 5}.

How many outcomes are in this event?

Compute it first, then check your number.

Hint

Count the listed outcomes in the event.

Solution

The event contains 1, 3, and 5, so it has 3 outcomes. An event can contain several outcomes, not just one.

MATH-C07-T02-003Exercise: Add probabilities for an event

A classifier assigns:

cat: 0.7
dog: 0.2
car: 0.1

What is the probability of the event "animal" = {cat, dog}?

Compute it first, then check your number.

Hint

Add the probabilities of cat and dog.

Solution

P(animal) = P(cat) + P(dog) = 0.7 + 0.2 = 0.9. The classifier assigns unequal probabilities, so the event probability comes from adding mass.

MATH-C07-T02-004Exercise: Sample space or event

In {cat, dog, car}, is {cat, dog} the whole sample space or an event?

Answer it first, then check.

Hint

It is a subset of all possible labels.

Solution

{cat, dog} is an event because it is a subset of the full sample space {cat, dog, car}.

MATH-C07-T02-005Exercise: Do not count unequal outcomes

A classifier assigns:

cat: 0.7
dog: 0.2
car: 0.1

Enter 1 if the probability of {cat, dog} is found by adding 0.7 + 0.2, not by counting two labels out of three.

Compute it first, then check your number.

Hint

The labels do not have equal probability.

Solution

Enter 1. Since the labels have unequal probabilities, the event probability is 0.7 + 0.2 = 0.9. Counting labels would ignore the model's distribution.

Before Moving On

An event is a set of outcomes. Probability measures the event.