How to remove duplicates from a list?

Question

I want to remove duplicates from a list but what I am doing is not working:

List<Customer> listCustomer = new ArrayList<Customer>();    
for (Customer customer: tmpListCustomer)
{
  if (!listCustomer.contains(customer)) 
  {
    listCustomer.add(customer);
  }
 }

Answer 1

IMHO best way how to do it these days:

Suppose you have a Collection "dups" and you want to create another Collection containing the same elements but with all duplicates eliminated. The following one-liner does the trick.

Collection<collectionType> noDups = new HashSet<collectionType>(dups);

It works by creating a Set which, by definition, cannot contain duplicates.

Based on oracle doc.

Answer 2

The cleanest way is:

List<XXX> lstConsultada = dao.findByPropertyList(YYY);
List<XXX> lstFinal = new ArrayList<XXX>(new LinkedHashSet<GrupoOrigen>(XXX));

and override hascode and equals over the Id's properties of each entity

Answer 3

List → Set → List (distinct)

Just add all your elements to a Set: it does not allow it's elements to be repeated. If you need a list afterwards, use new ArrayList(theSet) constructor afterwards (where theSet is your resulting set).

Answer 4

private void removeTheDuplicates(List<Customer>myList) {
    for(ListIterator<Customer>iterator = myList.listIterator(); iterator.hasNext();) {
        Customer customer = iterator.next();
        if(Collections.frequency(myList, customer) > 1) {
            iterator.remove();
        }
    }
    System.out.println(myList.toString());

}

Answer 5

Does Customer implement the equals() contract?

If it doesn't implement equals() and hashCode(), then listCustomer.contains(customer) will check to see if the exact same instance already exists in the list (By instance I mean the exact same object--memory address, etc). If what you are looking for is to test whether or not the same Customer( perhaps it's the same customer if they have the same customer name, or customer number) is in the list already, then you would need to override equals() to ensure that it checks whether or not the relevant fields(e.g. customer names) match.

Note: Don't forget to override hashCode() if you are going to override equals()! Otherwise, you might get trouble with your HashMaps and other data structures. For a good coverage of why this is and what pitfalls to avoid, consider having a look at Josh Bloch's Effective Java chapters on equals() and hashCode() (The link only contains iformation about why you must implement hashCode() when you implement equals(), but there is good coverage about how to override equals() too).

By the way, is there an ordering restriction on your set? If there isn't, a slightly easier way to solve this problem is use a Set<Customer> like so:

Set<Customer> noDups = new HashSet<Customer>();
noDups.addAll(tmpListCustomer);
return new ArrayList<Customer>(noDups);

Which will nicely remove duplicates for you, since Sets don't allow duplicates. However, this will lose any ordering that was applied to tmpListCustomer, since HashSet has no explicit ordering (You can get around that by using a TreeSet, but that's not exactly related to your question). This can simplify your code a little bit.

Answer 6

Using java 8 stream api.

    List<String> list = new ArrayList<>();
    list.add("one");
    list.add("one");
    list.add("two");
    System.out.println(list);
    Collection<String> c = list.stream().collect(Collectors.toSet());
    System.out.println(c);

Output:

Before values : [one, one, two]

After Values : [one, two]