7.4. White Space

We also have dealt with embedded white space before, using the two routines IsWhite and SkipWhite. Make sure that these routines are in your current version of the cradle, and add the the line

     SkipWhite;

at the end of both GetName and GetNum.

Now, let's define the new procedure:

{ Lexical Scanner }
Function Scan: string;
begin
   if IsAlpha(Look) then
      Scan := GetName
   else if IsDigit(Look) then
      Scan := GetNum
   else begin
      Scan := Look;
      GetChar;
   end;
   SkipWhite;
end;

We can call this from the new main program:

{ Main Program }
begin
   Init;
   repeat
      Token := Scan;
      writeln(Token);
   until Token = CR;
end.

Note

You will have to add the declaration of the string Token at the beginning of the program. Make it any convenient length, say 16 characters.

Now, run the program. Note how the input string is, indeed, separated into distinct tokens.