More descriptive tokens

Thus far the tokens that the parser has been dealing with are pretty generic - the title data, author data, etc are all symbolized with the FIELD token.

Let's modify our lexer so that it recognizes a token as Title data, and it recognizes a token as Author data, etc. For this, we will need to use states.

Thus far the tokens that the parser has been dealing with are pretty generic - the title data, author data, etc are all symbolized with the FIELD token.