Construct a lexical analyzer

. Allow white spaces, numbers and arithmetic operators in an expression

. Return tokens and attributes to the syntax analyzer

. A global variable tokenval is set to the value of the number

. Design requires that

- A finite set of tokens be defined

- Describe strings belonging to each token

We now try to construct a lexical analyzer for a language in which white spaces, numbers and arithmetic operators in an expression are allowed. From the input stream, the lexical analyzer recognizes the tokens and their corresponding attributes and returns them to the syntax analyzer. To achieve this, the function returns the corresponding token for the lexeme and sets a global variable, say tokenval , to the value of that token. Thus, we must define a finite set of tokens and specify the strings belonging to each token. We must also keep a count of the line number for the purposes of reporting errors and debugging. We will have a look at a typical code snippet which implements a lexical analyzer in the subsequent slide.