Python: How to set local variable in list comprehe

Question

I have a method that take a list and return an object

# input a list, returns an object
def map_to_obj(lst):
    a_list = f(lst)
    return a_list[0] if a_list else None

I want to get a list that contains all the mapped elements that isn't None.

Like this:

v_list = [v1, v2, v3, v4]

[map_to_obj(v) for v in v_list if map_to_obj(v)]

But it seems not good to call the map_to_obj method twice in the list comprehension.

Is there a way to have local variables in list comprehension so that it can have better performance?

Or does the compiler optimize it automatically?

Here is what I want:

(sml like)
[let mapped = map_to_obj(v) in for v in v_list if mapped end] 

Answer 1

List comprehensions are fine for the simple cases, but sometimes a plain old for loop is the simplest solution:

other_list = []
for v in v_list:
    obj = map_to_obj(v)
    if obj:
        other_list.append(obj)

Now if you really want a list comp and dont want to build an tmp list, you can use the iterator versions of filter and map:

import itertools as it
result = list(it.ifilter(None, it.imap(map_to_obj, v_list)))

or more simply :

import itertools as it
result = filter(None, it.imap(map_to_obj, v_list)))

The iterator versions don't build a temporary list, they use lazy evaluation.

Answer 2

A local variable can be set within a comprehension by cheating a bit and using an extra 'for' which "iterates" through a 1-element tuple containing the desired value for the local variable. Here's a solution to the OP's problem using this approach:

[o for v in v_list for o in (map_to_obj(v),) if o]

Here, o is the local variable being set equal to map_to_obj(v) for each v.

In my tests this is slightly faster than Lying Dog's nested generator expression (and also faster than the OP's double-call to map_to_obj(v), which, surprisingly, can be faster than the nested generator expression if the map_to_obj function isn't too slow).

Answer 3

Use nested list comprehension:

[x for x in [map_to_obj(v) for v in v_list] if x]

or better still, a list comprehension around a generator expression:

[x for x in (map_to_obj(v) for v in v_list) if x]

Answer 4

I have figured out a way of using reduce:

def map_and_append(lst, v):
    mapped = map_to_obj(v)
    if mapped is not None:
        lst.append(mapped)
    return lst

reduce(map_and_append, v_list, [])

How about the performance of this?

Answer 5

You can avoid re-calculation by using python built-in filter:

list(filter(lambda t: t is not None, map(map_to_obj, v_list)))

Answer 6

A variable assignment is just a singular binding:

[x   for v in l   for x in [v]]

This is a more general answer and also closer to what you proposed. So for your problem you can write:

[x   for v in v_list   for x in [map_to_obj(v)]   if x]