Get unique values from a list in python [duplicate

Question

This question already has an answer here:

• Removing duplicates in lists 43 answers

I want to get the unique values from the following list:

[u'nowplaying', u'PBS', u'PBS', u'nowplaying', u'job', u'debate', u'thenandnow']

The output which I require is:

[u'nowplaying', u'PBS', u'job', u'debate', u'thenandnow']

This code works:

output = []
for x in trends:
    if x not in output:
        output.append(x)
print output

is there a better solution I should use?

Answer 1

My solution to check contents for uniqueness but preserve the original order:

def getUnique(self):
    notunique = self.readLines()
    unique = []
    for line in notunique: # Loop over content
        append = True # Will be set to false if line matches existing line
        for existing in unique:
            if line == existing: # Line exists ? do not append and go to the next line
                append = False
                break # Already know file is unique, break loop
        if append: unique.append(line) # Line not found? add to list
    return unique

Edit: Probably can be more efficient by using dictionary keys to check for existence instead of doing a whole file loop for each line, I wouldn't use my solution for large sets.

Answer 2

Try this function, it's similar to your code but it's a dynamic range.

def unique(a):

    k=0
    while k < len(a):
        if a[k] in a[k+1:]:
            a.pop(k)
        else:
            k=k+1



    return a

Answer 3

First thing, the example you gave is not a valid list.

example_list = [u'nowplaying',u'PBS', u'PBS', u'nowplaying', u'job', u'debate',u'thenandnow']

Suppose if above is the example list. Then you can use the following recipe as give the itertools example doc that can return the unique values and preserving the order as you seem to require. The iterable here is the example_list

from itertools import ifilterfalse

def unique_everseen(iterable, key=None):
    "List unique elements, preserving order. Remember all elements ever seen."
    # unique_everseen('AAAABBBCCDAABBB') --> A B C D
    # unique_everseen('ABBCcAD', str.lower) --> A B C D
    seen = set()
    seen_add = seen.add
    if key is None:
        for element in ifilterfalse(seen.__contains__, iterable):
            seen_add(element)
            yield element
    else:
        for element in iterable:
            k = key(element)
            if k not in seen:
                seen_add(k)
                yield element

Answer 4

In addition to the previous answers, which say you can convert your list to set, you can do that in this way too

mylist = [u'nowplaying', u'PBS', u'PBS', u'nowplaying', u'job', u'debate', u'thenadnow']
mylist = [i for i in set(mylist)]

output will be

[u'nowplaying', u'job', u'debate', u'PBS', u'thenadnow']

though order will not be preserved.

Another simpler answer could be (without using sets)

>>> t = [v for i,v in enumerate(mylist) if mylist.index(v) == i]
[u'nowplaying', u'PBS', u'job', u'debate', u'thenadnow']

Answer 5

1. At the begin of your code just declare your output list as empty: output=[]
2. Instead of your code you may use this code trends=list(set(trends))

Answer 6

def get_distinct(original_list):
    distinct_list = []
    for each in original_list:
        if each not in distinct_list:
            distinct_list.append(each)
    return distinct_list