Remove duplicate elements from a list in Erlang

问题:

How can I remove the duplicate from a list in Erlang?

Suppose I have a list like:

[1,1,2,3,4,5,5,6]

How can I get:

[1,2,3,4,5,6]

回答1:

You could use sets, for example:

my_nonDuplicate_list1() ->
    List = [1,1,2,3,4,5,5,6],
    Set = sets:from_list(List),
    sets:to_list(Set).

This returns [1,2,3,4,5], no more duplicates, but most likely not sorted.

Another possibility without the usage of sets would be:

my_nonDuplicate_list2() ->
    List = [1,1,2,3,4,5,5,6],
    lists:usort(List).

In this case it returns [1,2,3,4,5], no more duplicates and sorted.

回答2:

And for those looking to preserve the order of the list:

remove_dups([])    -> [];
remove_dups([H|T]) -> [H | [X || X <- remove_dups(T), X /= H]].

回答3:

A possible solution that will Preserve the order of the elements to help you learn how to manipulate lists, would involve two functions:

delete_all(Item, [Item | Rest_of_list]) ->
    delete_all(Item, Rest_of_list);
delete_all(Item, [Another_item| Rest_of_list]) ->
    [Another_item | delete_all(Item, Rest_of_list)];
delete_all(_, []) -> [].

remove_duplicates(List)-> removing(List,[]). 
removing([],This) -> lists:reverse(This);
removing([A|Tail],Acc) -> 
    removing(delete_all(A,Tail),[A|Acc]).

To test,

Eshell V5.9  (abort with ^G)
1> mymod:remove_duplicates([1,2,3,1,2,4,1,2,1]).
[1,2,3,4]
2>

回答4:

I would do something like this at first to preserve order, though it is not recommended. Remember that AddedStuff ++ Accumulator is OK but Accumulator ++ AddedStuff is really bad.

rm_dup(List) ->
    lists:foldl(
        fun(Elem, Acc) ->
            case lists:member(Elem, Acc) of
                true ->
                    Acc;
                false ->
                    Acc ++ [Elem]
            end
        end, [], List
    ).

This solution is much more efficient if you want to preserve order:

rm_dup(List) ->
    lists:reverse(lists:foldl(
        fun(Elem, Acc) ->
            case lists:member(Elem, Acc) of
                true ->
                    Acc;
                false ->
                    [Elem] ++ Acc
            end
        end, [], List
    )).

回答5:

for my opinion, the best option is to use lists:usort()

But in case you don't want to use BIF's, and you want the list to be sorted, I suggest a version of quick sort, in this implementation you will get the list sorted without duplicate values.

unique_sort([]) -> [];
unique_sort([Pivot|T]) ->
unique_sort ([X || X <- T, X < Pivot ) ]++
[Pivot] ++
unique_sort ([X || X <- T, X > Pivot ]).

回答6:

Module sets has two functions that can be composed and do the job in an efficient way: sets:from_list/1 returns a set with all the elements of a list (with no duplicated elements from definition) and sets:to_list/1 returns a list with the elements of a set. Here is an example of use:

4> sets:to_list(sets:from_list([1,1,2,3,4,5,5,6])).
[3,6,2,5,1,4]

We could define the function as

nub(L) -> sets:to_list(sets:from_list(L)).