Conclusion

Count-based language models make prediction visible. They count contexts, count continuations, and estimate probabilities.

They also expose the central difficulty: language has too many possible contexts. More context can help, but longer contexts make the count table sparse.

The next chapter studies smoothing and backoff, which are ways to avoid treating unseen events as impossible.