Similarity After Learning

Embedding similarity is meaningful only after training gives the vectors useful structure.

At initialization, embedding rows are usually random or small values. Nearby points may not mean much yet. During training, rows that help with similar predictions may receive similar updates.

A common similarity measure is cosine similarity:

cosine_similarity(a, b) = dot(a, b) / (||a|| ||b||)

For simple unit vectors:

a = [1, 0]
b = [0, 1]
dot(a, b) = 0
cosine similarity = 0

This says the directions are orthogonal, not similar in direction.

querynearnearnearfarfarneighborhooddistance is useful, not magic
Embedding geometry studies neighborhoods, directions, and boundaries.

Similarity is evidence, not a final explanation. Always connect it back to the data and task.

Exercise: Dot product similarity

Let a = [1, 2] and b = [3, 4]. What is a dot b?

Compute it first, then check your number.

Exercise: Orthogonal cosine

Unit vectors [1, 0] and [0, 1] have what cosine similarity?

Compute it first, then check your number.