my lexer token action is not invoked

2019-07-24 20:14发布

问题:

I use antlr4 with javascript target.

Here is a sample grammar:

P : T ;
T : [a-z]+ {console.log(this.text);} ;
start: P ;

When I run the generated parser, nothing is printed, although the input is matched. If I move the action to the token P, then it gets invoked. Why is that?

回答1:

Actions are ignored in referenced rules. This was the original behavior of ANTLR 4, back when the lexer only supported a single action per token (and that action must appear at the end of the token).

Several releases later the limitation of one-action-per-rule was lifted, allowing any number of actions to be executed for a token. However, we found that many existing users relied on the original behavior, and wrote their grammars assuming that actions in referenced rules were ignored. Many of these grammars used complicated logic in these rules, so changing the behavior would be a severe breaking change that would prevent people from using new versions of ANTLR 4.

Rather than break so many existing ANTLR 4 lexers, we decided to preserve the original behavior and only execute actions that appear in the same rule as the matched token. Newer versions do allow you to place multiple actions in each rule though.

tl;dr: We considered allowing actions in other rules to execute, but decided not to because it would break a lot of grammars already written and used by people.



回答2:

I found that @init and @after actions will override this default behavior. Change the example code to:

grammar Test;

ALPHA : [a-z]+;

p : t ;

t
@init {
    console.log(this.text);
}
@after { 
    console.log(this.text);
}  
:  ALPHA;
start: p ;

I changed parser rules to LOWER case as my Eclipse tool was complaining about the syntax otherwise. I also had to insert ALPHA for [a-z]+; for the same reason. The above text compiled, but I haven't tried running the generated parser. However, I am successfully working around this issue with @init/@after in my larger parser.

Hope this is helpful.



标签: antlr antlr4