| # The Token stream | |
| The token stream is a doubly linked list with special 'EOF' tokens at the | |
| beginning and end. The EOF token at the end points to itself as the `next` | |
| token, and the one at the beginning points to itself as the `previous` token. | |
| So, for a correctly formed token stream (which we routinely assume will always | |
| be the case) neither `previous` nor `next` can ever return `null`. We can't | |
| express that in the type system because those fields will be `null` temporarily | |
| while the list is being built. As a result, the convention throughout the code | |
| base, is to suffix those two getters with a bang (`!`) everywhere they're | |
| invoked. | |
| Tokens are typically accessed through the [AST](ast.md). |