How to find all duplicate from a List?

Question

I have a List<string> which has some words duplicated. I need to find all words which are duplicates.

Any trick to get them all?

Answer 1

and without the LINQ:

string[] ss = {"1","1","1"};

var myList = new List<string>();
var duplicates = new List<string>();

foreach (var s in ss)
{
   if (!myList.Contains(s))
      myList.Add(s);
   else
      duplicates.Add(s);
}

// show list without duplicates 
foreach (var s in myList)
   Console.WriteLine(s);

// show duplicates list
foreach (var s in duplicates)
   Console.WriteLine(s);

Answer 2

If you are using LINQ, you can use the following query:

var duplicateItems = from x in list
                     group x by x into grouped
                     where grouped.Count() > 1
                     select grouped.Key;

or, if you prefer it without the syntactic sugar:

var duplicateItems = list.GroupBy(x => x).Where(x => x.Count() > 1).Select(x => x.Key);

This groups all elements that are the same, and then filters to only those groups with more than one element. Finally it selects just the key from those groups as you don't need the count.

If you're prefer not to use LINQ, you can use this extension method:

public void SomeMethod {
    var duplicateItems = list.GetDuplicates();
    …
}

public static IEnumerable<T> GetDuplicates<T>(this IEnumerable<T> source) {
    HashSet<T> itemsSeen = new HashSet<T>();
    HashSet<T> itemsYielded = new HashSet<T>();

    foreach (T item in source) {
        if (!itemsSeen.Add(item)) {
            if (itemsYielded.Add(item)) {
                yield return item;
            }
        }
    }
}

This keeps track of items it has seen and yielded. If it hasn't seen an item before, it adds it to the list of seen items, otherwise it ignores it. If it hasn't yielded an item before, it yields it, otherwise it ignores it.

Answer 3

Using LINQ, ofcourse. The below code would give you dictionary of item as string, and the count of each item in your sourc list.

var item2ItemCount = list.GroupBy(item => item).ToDictionary(x=>x.Key,x=>x.Count());

Answer 4

    lblrepeated.Text = ""; 
    string value = txtInput.Text;
    char[] arr = value.ToCharArray();
    char[] crr=new char[1];        
   int count1 = 0;        
    for (int i = 0; i < arr.Length; i++)
    {
        int count = 0;  
        char letter=arr[i];
        for (int j = 0; j < arr.Length; j++)
        {
            char letter3 = arr[j];
                if (letter == letter3)
                {
                    count++;
                }                    
        }
        if (count1 < count)
        {
            Array.Resize<char>(ref crr,0);
            int count2 = 0;
            for(int l = 0;l < crr.Length;l++)
            {
                if (crr[l] == letter)
                    count2++;                    
            }


            if (count2 == 0)
            {
                Array.Resize<char>(ref crr, crr.Length + 1);
                crr[crr.Length-1] = letter;
            }

            count1 = count;               
        }
        else if (count1 == count)
        {
            int count2 = 0;
            for (int l = 0; l < crr.Length; l++)
            {
                if (crr[l] == letter)
                    count2++;
            }


            if (count2 == 0)
            {
                Array.Resize<char>(ref crr, crr.Length + 1);
                crr[crr.Length - 1] = letter;
            }

            count1 = count; 
        }
    }

    for (int k = 0; k < crr.Length; k++)
        lblrepeated.Text = lblrepeated.Text + crr[k] + count1.ToString();

Answer 5

If you're looking for a more generic method:

public static List<U> FindDuplicates<T, U>(this List<T> list, Func<T, U> keySelector)
    {
        return list.GroupBy(keySelector)
            .Where(group => group.Count() > 1)
            .Select(group => group.Key).ToList();
    }

EDIT: Here's an example:

public class Person {
    public string Name {get;set;}
    public int Age {get;set;}
}

List<Person> list = new List<Person>() { new Person() { Name = "John", Age = 22 }, new Person() { Name = "John", Age = 30 }, new Person() { Name = "Jack", Age = 30 } };

var duplicateNames = list.FindDuplicates(p => p.Name);
var duplicateAges = list.FindDuplicates(p => p.Age);

foreach(var dupName in duplicateNames) {
    Console.WriteLine(dupName); // Will print out John
}

foreach(var dupAge in duplicateAges) {
    Console.WriteLine(dupAge); // Will print out 30
}

Answer 6

I'm assuming each string in your list contains several words, let me know if that's incorrect.

List<string> list = File.RealAllLines("foobar.txt").ToList();

var words = from line in list
            from word in line.Split(new[] { ' ', ';', ',', '.', ':', '(', ')' }, StringSplitOptions.RemoveEmptyEntries)
            select word;

var duplicateWords = from w in words
                     group w by w.ToLower() into g
                     where g.Count() > 1
                     select new
                     {
                         Word = g.Key,
                         Count = g.Count()
                     }