Does regex lookahead affect subsequent match?

Question

I was playing around with regular expression look-aheads and came across something I don't understand.

I would expect this regular expression:

(?=1)x

to match this string:

"x1"

But it doesn't. In ruby the code looks like:

> "x1".match /(?=1)x/
=> nil

Here's what I would expect to happen:

1. We start with the regular expression parser's cursor on "x".
2. The regexp engine searches the string for "1" and gets a match. The cursor is still on "x"
3. The regexp engine searches for "x" and finds it, since the cursor hasn't moved.
4. Success! Profit!

But I'm apparently mistaken, because it's not matching. Could someone please tell me where I've gone wrong?

Incidentally, I've noticed that if the pattern matched by the lookahead contains the characters I'm matching in the subsequent expression, it works. ie. (?=x)x matches x1 just fine. I suspect this is the key to the mystery, but I'm just not getting it. :)

Answer 1

A look-ahead does not move the regex index forward, it "stands its ground", but it requires presence or absence of some pattern after the current position in string.

When you use (?=1)x, you tell the regex engine:

1. The next character must be 1
2. Right at this position, match the character x.

It means you require x to be 1 which is never true/is always false. This regex will never match anything.

Here is another example from regular-expressions.com:

Let's apply q(?=u)i to quit. The lookahead is now positive and is followed by another token. Again, q matches q and u matches u. Again, the match from the lookahead must be discarded, so the engine steps back from i in the string to u. The lookahead was successful, so the engine continues with i. But i cannot match u. So this match attempt fails. All remaining attempts fail as well, because there are no more q's in the string.

Another must-read resource is Lookarounds Stand their Ground at rexegg.com:

Lookahead and lookbehind don't mean look way ahead into the distance. They mean look at the text immediately to the left or to the right. If you want to inspect a piece of string further down, you will need to insert "binoculars" inside the lookahead to get you to the part of the string you want to inspect—for instance a .*, or, ideally, more specific tokens.

And

Do not expect the pattern A(?=5) to match the A in the string AB25. Many beginners assume that the lookahead says that "there is a 5 somewhere to the right", but that is not so. After the engine matches the A, the lookahead (?=5) asserts that at the current position in the string, what immediately follows is a 5. If you want to check if there is a 5 somewhere (anywhere) to the right, you can use (?=[^5]*5).

Answer 2

I'm not going to give you a long dissertation about regular expression assertions.

But I will tell you how to never confuse what they are, nor ever forget how to use them.

Regular expressions are processed (parsed) left to right.
They are nothing more than a fancy template.

ASSERTIONS exist BETWEEN characters in the target text, just like they exist
between expressions in the regular expression.

They don't exist AT characters, but between them.

That means you could easily look left or right and apply an appropriate
assertion, i.e. lookAHEAD or lookBEHIND.

That's all you really need to know to get started.

---

Your regex (?=1)x for example:

The regex says at a position between characters look ahead for a 1,
if it looked and found a 1, continue with the next expression.
The next expression is looking for a literal x.

Now, if the next character is 1, then its not x.
Result is, the regex bomb's, it can never match anything.