ANTLR: Get text representation of a sub lexer rule

问题:

consider the following lexer rules in ANTLR4:

ID: [a-z]+;
INT: [0-9]+;
ARRAY: ID '[' INT ']';

Is it possible in a tree walking scenario where I have access to ctx.ARRAY() (where ctx is a subclass of ParserRuleContext that was generated out of a parser rule) to get the text representation of the lexer rules ID and INT? I currently fetch the whole text representation with ctx.ARRAY().getText() and parse the contents of ID and INT using regexes and was just wondering if there is a 'cleaner' out of the box solution ANTLR provides.

Note: Because of external dependencies making ARRAY a parser rule is not an option.

Thanks in advance for meaningful answers.

回答1:

Lexer rules in ANTLR 4 cannot be broken down into parts. This was a design decision that we made as part of a massive speed and memory improvement for ANTLR 4 lexers over ANTLR 3 lexers. ANTLR 3 lexers were recursive descent recognizers with many of the same features as parsers. In ANTLR 4, the lexer is nothing more than a DFA recognizer with support for semantic predicates, so the boundaries between the individual components of a token are not tracked at all.

You'll have to either make ARRAY a parser rule, or separately parse the result of getText() when you need to break up the token's text.