Embedding Geometry
An embedding maps an object into a vector space.
The object might be a word, token, image patch, user, item, document, or state. After embedding, the object becomes a vector that can be compared with other vectors.
What Geometry Gives Us
Embedding geometry lets us ask:
- Which points are near this point?
- Which directions correspond to useful changes?
- Which clusters or neighborhoods appear?
- Which boundaries separate categories?
These questions are useful, but they need care.
A query embedding is closest to point A, farther from point B, and farthest from point C.
Which point would a nearest-neighbor search return?
Compute it first, then check your number.
HintClosest point
Nearest-neighbor search returns the closest point under the chosen distance.
SolutionNearest neighbor
Nearest-neighbor search returns the point with the smallest distance under
the chosen measure. The prompt says point A is closest, so it would return
point A. Enter 1.
Nearby Does Not Always Mean Explained
If two embeddings are near each other, the model has placed them near each other under its learned representation. That can suggest a relationship, but it is not a full explanation.
The geometry is evidence. It is not a guarantee.
Enter 1 if this statement is too strong:
If two embeddings are nearby,
we fully understand why
the model placed them there.
Compute it first, then check your number.
HintEvidence, not proof
Nearby embeddings may suggest a relationship, but that is not the same as a complete explanation.
SolutionCareful interpretation
Enter 1. Embedding geometry helps inspect representations, but closeness
alone does not fully explain the model's internal reason.
Direction Can Carry Meaning
Sometimes a direction in embedding space corresponds to a useful change. For example, a direction may move representations toward a feature, topic, style, or class.
This is one reason projections, dot products, and cosine similarity keep appearing in interpretability work.
But a direction should earn its interpretation. A plotted direction, nearest neighbor, or analogy can suggest a hypothesis. It becomes more trustworthy when you test it on many examples, compare it with baselines, and check that the effect is not an artifact of one prompt or one visualization.
Enter 1 if cosine similarity is often used to compare embedding directions,
or 2 if it is used to count the number of coordinates.
Compute it first, then check your number.
HintDirection comparison
Cosine similarity comes from the angle between nonzero vectors.
SolutionCosine for direction
Enter 1. Cosine similarity compares directions after accounting for
vector lengths. It is a natural tool when the angle between embeddings
matters more than their raw magnitude.
Enter 1 if a claimed embedding direction should be tested across examples,
not accepted from one visualization alone.
Compute it first, then check your number.
HintHypothesis first
Treat the direction as a hypothesis about the representation.
SolutionEvidence across examples
Enter 1. A single plot can suggest a direction, but the interpretation is
stronger when the direction behaves consistently across examples.
Before Moving On
Embedding geometry is a way to inspect representations, not a promise that the space will be simple.