Introduction

Neural language models replace explicit count tables with learned parameters.

The task is still familiar:

context -> probability distribution over next tokens

The difference is how the distribution is produced. Token ids become embeddings. The embeddings pass through a neural model. The final scores become probabilities with softmax.

A training step measures error, computes gradients, updates parameters, and repeats on another batch.

What this chapter covers

neural next-token prediction;
input embeddings and output scores;
softmax over a vocabulary;
training with cross-entropy;
why neural models generalize beyond exact counts.