Python list comprehension, with unique items

Question

Is there a way to make a list comprehension in Python that only contains unique items?

My original idea was to use something like this : new_items = [unicode(item) for item in items]

However, I later realized that I needed to omit duplicate items. So I ended up with this ugly monstrosity :

unique_items = []
for item in items :
    unicode_item = unicode(item)
    if unicode_item not in unique_items :
        unique_items.append(unicode_item)

Now this is far less pretty (and readable) than a simple list comprehension. So, is there a way to make a list comprehension equivalent to the above code?

Also order does matter, so I can't just use a set comprehension.

Answer 1

Make it a helper function, like so.

def unique_iter(iterable):
  seen = set()
  for item in iterable:
    if item in seen:
      continue
    seen.add(item)
    yield item

for ch in unique_iter("never gonna give you up"):
  print ch,

outputs

n e v r g o a i y u p

Answer 2

Your original idea works with a set comprehension:

new_items = {unicode(item) for item in items}

Answer 3

I short one liner might be:

s = "some string"
unique_items = [unicode(ch) for ch in sorted(set(s), key=s.index)]

Answer 4

Well, there is no ordered set, but we can misuse OrderedDict:

from collections import OrderedDict
t = "never gonna give you up"
OrderedDict.fromkeys(t).keys()

Gives:

['n', 'e', 'v', 'r', ' ', 'g', 'o', 'a', 'i', 'y', 'u', 'p']