Recursive expressions with pyparsing

Question

I'm trying to figure out how to do a left-associative expression where recursive (not-enclosed in anything) expressions are possible. For example, I'd like to do:

expr + OP + expr

that parses 2 operations like 1 x 2 x 3 into (expr OP expr) OP expr result.

If I try to prevent expr parsing from infinite recursion, i can do something like:

expr -> Group(simple_expr + OP + expr)
      | simple_expr

but then I'd get the expr OP (expr OR expr) result.

How do I force left-side binding?

Edit: I know about the operatorPrecedence but when the operator is "IS" + Optional("NOT") or similar, it doesn't seem to match properly.

Answer 1

Here is an example parse action that will take the flat lists of tokens and nest them as if parsed left-recursively:

from pyparsing import *

# parse action -maker
def makeLRlike(numterms):
    if numterms is None:
        # None operator can only by binary op
        initlen = 2
        incr = 1
    else:
        initlen = {0:1,1:2,2:3,3:5}[numterms]
        incr = {0:1,1:1,2:2,3:4}[numterms]

    # define parse action for this number of terms,
    # to convert flat list of tokens into nested list
    def pa(s,l,t):
        t = t[0]
        if len(t) > initlen:
            ret = ParseResults(t[:initlen])
            i = initlen
            while i < len(t):
                ret = ParseResults([ret] + t[i:i+incr])
                i += incr
            return ParseResults([ret])
    return pa


# setup a simple grammar for 4-function arithmetic
varname = oneOf(list(alphas))
integer = Word(nums)
operand = integer | varname

# ordinary opPrec definition
arith1 = operatorPrecedence(operand,
    [
    (None, 2, opAssoc.LEFT),
    (oneOf("* /"), 2, opAssoc.LEFT),
    (oneOf("+ -"), 2, opAssoc.LEFT),
    ])

# opPrec definition with parseAction makeLRlike
arith2 = operatorPrecedence(operand,
    [
    (None, 2, opAssoc.LEFT, makeLRlike(None)),
    (oneOf("* /"), 2, opAssoc.LEFT, makeLRlike(2)),
    (oneOf("+ -"), 2, opAssoc.LEFT, makeLRlike(2)),
    ])

# parse a few test strings, using both parsers
for arith in (arith1, arith2):
    print arith.parseString("A+B+C+D+E")[0]
    print arith.parseString("A+B+C*D+E")[0]
    print arith.parseString("12AX+34BY+C*5DZ+E")[0]

Prints:

(normal)

['A', '+', 'B', '+', 'C', '+', 'D', '+', 'E']
['A', '+', 'B', '+', ['C', '*', 'D'], '+', 'E']
[['12', 'A', 'X'], '+', ['34', 'B', 'Y'], '+', ['C', '*', ['5', 'D', 'Z']], '+', 'E']

(LR-like)

[[[['A', '+', 'B'], '+', 'C'], '+', 'D'], '+', 'E']
[[['A', '+', 'B'], '+', ['C', '*', 'D']], '+', 'E']
[[[[['12', 'A'], 'X'], '+', [['34', 'B'], 'Y']], '+', ['C', '*', [['5', 'D'], 'Z']]], '+', 'E']

Answer 2

Pyparsing produces left parse trees. Add a semantic action to edit the parse tree right after expr has been parsed.