removing duplicates of a list of sets

Question

I have a list of sets :

L = [set([1, 4]), set([1, 4]), set([1, 2]), set([1, 2]), set([2, 4]), set([2, 4]), set([5, 6]), set([5, 6]), set([3, 6]), set([3, 6]), set([3, 5]), set([3, 5])]

(actually in my case a conversion of a list of reciprocal tuples)

and I want to remove duplicates to get :

L = [set([1, 4]), set([1, 2]), set([2, 4]), set([5, 6]), set([3, 6]), set([3, 5])]

But if I try :

>>> list(set(L))
TypeError: unhashable type: 'set'

Or

>>> list(np.unique(L))
TypeError: cannot compare sets using cmp()

How do I get a list of sets with distinct sets?

Answer 1

The best way is to convert your sets to frozensets (which are hashable) and then use set to get only the unique sets, like this

>>> list(set(frozenset(item) for item in L))
[frozenset({2, 4}),
 frozenset({3, 6}),
 frozenset({1, 2}),
 frozenset({5, 6}),
 frozenset({1, 4}),
 frozenset({3, 5})]

If you want them as sets, then you can convert them back to sets like this

>>> [set(item) for item in set(frozenset(item) for item in L)]
[{2, 4}, {3, 6}, {1, 2}, {5, 6}, {1, 4}, {3, 5}]

---

If you want the order also to be maintained, while removing the duplicates, then you can use collections.OrderedDict, like this

>>> from collections import OrderedDict
>>> [set(i) for i in OrderedDict.fromkeys(frozenset(item) for item in L)]
[{1, 4}, {1, 2}, {2, 4}, {5, 6}, {3, 6}, {3, 5}]

Answer 2

Here is another alternative

yourNewSet = map(set,list(set(map(tuple,yourSet))))

Answer 3

There is another alternative.

import itertools
list_sets = [set(['a', 'e', 'f']), set(['c', 'b', 'f']), set(['a', 'e', 'f']), set(['a', 'd']), set(['a', 'e', 'f'])]

lists = [list(s) for s in list_sets] # convert a list of sets to a list of lists
lists.sort()
lists_remove_duplicates = [lists for lists,_ in itertools.groupby(lists)]
print(lists_remove_duplicates)

# output
[['a', 'd'], ['a', 'e', 'f'], ['c', 'b', 'f']]

Answer 4

An alternative using a loop:

result = list()
for item in L:
    if item not in result:
        result.append(item)