Chapter 3: Lexical Analysis

PL/1 Problems

. Keywords are not reserved in PL/1 if then then then = else else else = then if if then then = then + 1

. PL/1 declarations

Declare(arg 1 ,arg 2 ,arg 3 ,...,arg n )

. Cannot tell whether Declare is a keyword or array reference until after " ) "

. Requires arbitrary lookahead and very large buffers . Worse, the buffers may have to be reloaded.

In many languages certain strings are reserved, i.e., there meaning is predefined and cannot be changed by the user. If keywords are not reserved then the lexical analyzer must distinguish between a keyword and a user defined identifier. PL/1 has several problems: 1. In PL/1 keywords are not reserved; thus, the rules for distinguishing keywords from identifiers are quite complicated as the following PL/1 statement illustrates. For example - If then then then = else else else = then 2. PL/1 declarations: Example - Declare (arg1, arg2, arg3,.., argn) In this statement, we can not tell whether 'Declare' is a keyword or array name until we see the character that follows the ")". This requires arbitrary lookahead and very large buffers. This buffering scheme works quite well most of the time but with it the amount of lookahead is limited and this limited lookahead may make it impossible to recognize tokens in salutations where the distance the forward pointer must travel is more than the length of the buffer, as the slide illustrates. The situation even worsens if the buffers have to be reloaded.