Chapter 3: Lexical Analysis

Interface to other phases

. Push back is required due to lookahead for example > = and >

. It is implemented through a buffer

- Keep input in a buffer

- Move pointers over the input

The lexical analyzer reads characters from the input and passes tokens to the syntax analyzer whenever it asks for one. For many source languages, there are occasions when the lexical analyzer needs to look ahead several characters beyond the current lexeme for a pattern before a match can be announced. For example, > and >= cannot be distinguished merely on the basis of the first character >. Hence there is a need to maintain a buffer of the input for look ahead and push back. We keep the input in a buffer and move pointers over the input. Sometimes, we may also need to push back extra characters due to this lookahead character.