How to 'flatten' generators in python?

Question

I have a problem with 'flattening' out some generators in python. Here is my code:

import itertools as it
test = [[1,2,3],[4,5],[6,7,8]]
def comb(possible):
    if len(possible) != 1:
        for a in possible[0]:
            yield from it.product((a,), comb(possible[1:]))
    else:
        yield from possible[0]
list(comb(test))

which gives me:

[(1, (4, 6)),
(1, (4, 7)),
(1, (4, 8)),
(1, (5, 6)),
(1, (5, 7)),
(1, (5, 8)),
(2, (4, 6)),
(2, (4, 7)),
(2, (4, 8)),
(2, (5, 6)),
(2, (5, 7)),
(2, (5, 8)),
(3, (4, 6)),
(3, (4, 7)),
(3, (4, 8)),
(3, (5, 6)),
(3, (5, 7)),
(3, (5, 8))]

However, I want something like:

[(1, 4, 6),
(1, 4, 7),
(1, 4, 8),
(1, 5, 6),
(1, 5, 7),
(1, 5, 8),
(2, 4, 6),
(2, 4, 7),
(2, 4, 8),
(2, 5, 6),
(2, 5, 7),
(2, 5, 8),
(3, 4, 6),
(3, 4, 7),
(3, 4, 8),
(3, 5, 6),
(3, 5, 7),
(3, 5, 8)]

In general the function should give me generators for all possible paths to go through a list, i.e. from test[0] -> test[1] -> ... -> test[n] where n is len(test). Here, it picks up at each step one element.

Similar to what the following function returns, just with generators:

def prod(possible):
    if len(possible) != 1:
        b = []
        for i in range(len(possible[0])):
            for x in prod(possible[1:]):
                if len(possible) == 2:
                    b += [[possible[0][i]]+[x]]
                else:
                    b += [[possible[0][i]]+x]
        return b
    else:
        return possible[0]
prod(test)

I played around with it.chain and it.chain.from_iterable but can't seem to make it work. The problem is that my 'test' list are variable in size and length and thus I have to do the whole thing recursively.

Edit:

itertools.product(*test)

works as pointed out by John Coleman

Answer 1

Here's one way to calculate a product of lists without using the built-in

def product (*iters):
  def loop (prod, first = [], *rest):
    if not rest:
      for x in first:
        yield prod + (x,)
    else:
      for x in first:
        yield from loop (prod + (x,), *rest)
  yield from loop ((), *iters)

for prod in product ("ab", "xyz"):
  print (prod)

# ('a', 'x')
# ('a', 'y')
# ('a', 'z')
# ('b', 'x')
# ('b', 'y')
# ('b', 'z')

In python, we can collect the outputs of a generator in a list by using the list constructor. Note we can also calculate the product of more than two inputs as seen below

print (list (product ("+-", "ab", "xyz")))
# [ ('+', 'a', 'x')
# , ('+', 'a', 'y')
# , ('+', 'a', 'z')
# , ('+', 'b', 'x')
# , ('+', 'b', 'y')
# , ('+', 'b', 'z')
# , ('-', 'a', 'x')
# , ('-', 'a', 'y')
# , ('-', 'a', 'z')
# , ('-', 'b', 'x')
# , ('-', 'b', 'y')
# , ('-', 'b', 'z')
# ]

Because product accepts a a list of iterables, any iterable input can be used in the product. They can even be mixed as demonstrated below

print (list (product (['@', '%'], range (2), "xy")))
# [ ('@', 0, 'x')
# , ('@', 0, 'y')
# , ('@', 1, 'x')
# , ('@', 1, 'y')
# , ('%', 0, 'x')
# , ('%', 0, 'y')
# , ('%', 1, 'x')
# , ('%', 1, 'y')
# ]

Because product is defined as a generator, we are afforded much flexibility even when writing more complex programs. Consider this program that finds right triangles made up whole numbers, a Pythagorean triple. Also note that product allows you to repeat an iterable as input as see in product (r, r, r) below

def is_triple (prod):
  (a,b,c) = prod
  return a * a + b * b == c * c

def solver (n):
  r = range (1,n)
  for p in product (r, r, r):
    if is_triple (p):
      yield p

print (list (solution in solver (20)))
# (3, 4, 5)
# (4, 3, 5)
# (5, 12, 13)
# (6, 8, 10)
# (8, 6, 10)
# (8, 15, 17)
# (9, 12, 15)
# (12, 5, 13)
# (12, 9, 15)
# (15, 8, 17)

For additional explanation and a way to see how to do this without using generators, view this answer.