RECOGNITION OF TOKENS

Boomi Nathan
1 Min Read
Disclosure: This website may contain affiliate links, which means I may earn a commission if you click on the link and make a purchase. I only recommend products or services that I personally use and believe will add value to my readers. Your support is appreciated!

Consider the following grammar fragment: stmt → if expr then stmt

| if expr then stmt else stmt | ε

expr → term relop term

| term

                           term → id | num

where the terminals if , then, else, relop, id and num generate sets of strings given by the following regular definitions:

if → if 

then → then

else → else

relop → <|<=|=|<>|>|>=

id → letter(letter|digit)*

num → digit+ (.digit+)?(E(+|-)?digit+)?

For this language fragment the lexical analyzer will recognize the keywords if, then, else, as well as the lexemes denoted by relop, id, and num. To simplify matters, we assume keywords are reserved; that is, they cannot be used as identifiers.

Transition diagrams

It is a diagrammatic representation to depict the action that will take place when a lexical analyzer is called by the parser to get the next token. It is used to keep track of information about the characters that are seen as the forward pointer scans the input.

Share This Article

J. BoomiNathan is a writer at SenseCentral who specializes in making tech easy to understand. He covers mobile apps, software, troubleshooting, and step-by-step tutorials designed for real people—not just experts. His articles blend clear explanations with practical tips so readers can solve problems faster and make smarter digital choices. He enjoys breaking down complicated tools into simple, usable steps.

Leave a review