Lexical analyzer performs below given tasks:
- Helps to identify token into the symbol table
- Removes white spaces and comments from the source program
- Correlates error messages with the source program
- Helps you to expands the macros if it is found in the source program
- Read input characters from the source program
Example of Lexical Analysis, Tokens, Non-Tokens
Consider the following code that is fed to Lexical Analyzer
#include <stdio.h>
int maximum(int x, int y) {
// This will compare 2 numbers
if (x > y)
return x;
else {
return y;
}
}
Examples of Tokens created
| Lexeme | Token |
| int | Keyword |
| maximum | Identifier |
| ( | Operator |
| int | Keyword |
| x | Identifier |
| , | Operator |
| int | Keyword |
| Y | Identifier |
| ) | Operator |
| { | Operator |
| If | Keyword |
Examples of Nontokens
| Type | Examples |
| Comment | // This will compare 2 numbers |
| Pre-processor directive | #include <stdio.h> |
| Pre-processor directive | #define NUMS 8,9 |
| Macro | NUMS |
| Whitespace | /n /b /t |