Why Long Dependencies Matter

Language often depends on earlier context.

In:

The book that the students borrowed yesterday was expensive.

the word was agrees with book, not with students.

A model that only remembers the most recent words may miss the structure. Long dependencies matter for agreement, reference, code blocks, stories, and multi-step instructions.

Exercise

In the sentence above, which word controls was: book or students? Answer 1 for book, 2 for students.