I need a regex in jFlex to match a string literal, containing some characters, followed by a hyphen which is followed by a word. However, there are a few hardcoded exceptions. My jFlex version is 1.6.1
My regexes are:
SUFFIXES = labeled|deficient
ALPHANUMERIC = [:letter:]|[:digit:]
AVOID_SUFFIXES = {SUFFIXES} | !({ALPHANUMERIC}+)
WORD = ({ALPHANUMERIC}+([\-\/\.]!{AVOID_SUFFIXES})*)
String "MXs12-labeled" should be tokenized into 'MXs12', '-', 'labeled' (hyphen caught by different regex later), and "MXs12-C123" into 'MXs12-C123' as C123 is not on list of suffixes.
However, the token I obtain is "MXs12-labele" - one letter short of the one forbidden by exception.
An obvious solution would be including additional non {ALPHANUMERIC} character in the regex, but that would add this character to the match too. Another solution seemed to be to use a negative lookahead, but they return a syntax error every time I try to parse them - jFlex seems not to supports it. (Flex seems do not support a regex lookahead assertion (the fast lex analyzer))
Does anyone know how to solve this in jFlex?
As you've observed, it's much easier to work with positive matches than with negative matches. (Clearly,
labele
does not matchlabeled
, and furthermore it's the longest prefix oflabeled
which doesn't matchlabeled
, so it's logical that if you try to match a word which is!labeled
, you'll getlabele
as a match.JFlex does not implement negative lookahead assertions, which are slightly different but still problematic. A negative lookahead assertion would certainly reject the suffix in
MXs12-labeled
, but it would also reject the suffix inMXs12-labeledblack
, which would be a bit surprising, I think.If you rephrase this with positive matches, though, it's really simple. The idea is to specify what needs to be done with every positive match. In this case, what we'll want to do with the positive match of
-labeled
is to put it back into the input stream, which can be done withyypushback
. That would suggest rules something like this:Note that order is important, since the sequence relies on the first two patterns having higher precedence than the last pattern. (Inputs which match one of the first two patterns will also match the last pattern, but with the rules in the order indicated the last pattern will not win.)
That might or might not be what you really want. It will handle
MXs12-labeled
andMXs12-C123
as indicated in your question.MXs12-labeledblack
andMXs12-labeled-black
will both be reported as single tokens; it's not at all clear to me what your expectations are on these inputs.Rici's answer solved the problem -
yypushback()
was exactly what I needed. As of nowyypushback()
if so.With additional java regex, I can cover the mentioned edge cases, e.g.
"\\-labeled$"
ensures that suffix is at the end of passed string andMXs12-labeled-black
will be returned as one token, whereasMXs12-labeled
as three. Thank you very much!