Best way to remove duplicate entries from a data t

Question

What is the best way to remove duplicate entries from a Data Table?

Answer 1

    /* To eliminate Duplicate rows */
    private void RemoveDuplicates(DataTable dt)
    {

        if (dt.Rows.Count > 0)
        {
            for (int i = dt.Rows.Count - 1; i >= 0; i--)
            {
                if (i == 0)
                {
                    break;
                }
                for (int j = i - 1; j >= 0; j--)
                {
                    if (Convert.ToInt32(dt.Rows[i]["ID"]) == Convert.ToInt32(dt.Rows[j]["ID"]) && dt.Rows[i]["Name"].ToString() == dt.Rows[j]["Name"].ToString())
                    {
                        dt.Rows[i].Delete();
                        break;
                    }
                }
            }
            dt.AcceptChanges();
        }
    }

Answer 2

There is a simple way using Linq GroupBy Method.

var duplicateValues = dt.AsEnumerable() 

        .GroupBy(row => row[0]) 

        .Where(group => (group.Count() == 1 || group.Count() > 1)) 

        .Select(g => g.Key); 



foreach (var d in duplicateValues)

        Console.WriteLine(d);

• See more at: http://www.dotnetlines.com/Blogs/tabid/85/EntryId/49/Remove-duplicate-rows-from-a-DataTable-using-LINQ.aspx

Answer 3

A simple way would be:

 var newDt= dt.AsEnumerable()
                 .GroupBy(x => x.Field<int>("ColumnName"))
                 .Select(y => y.First())
                 .CopyToDataTable();

Answer 4

Heres a easy and fast way using AsEnumerable().Distinct()

private DataTable RemoveDuplicatesRecords(DataTable dt)
{
    //Returns just 5 unique rows
    var UniqueRows = dt.AsEnumerable().Distinct(DataRowComparer.Default);
    DataTable dt2 = UniqueRows.CopyToDataTable();
    return dt2;
}

CLICK to visit my blog for more Detail

Answer 5

Remove Duplicates

public DataTable RemoveDuplicateRows(DataTable dTable, string colName)
{
   Hashtable hTable = new Hashtable();
   ArrayList duplicateList = new ArrayList();

   //Add list of all the unique item value to hashtable, which stores combination of key, value pair.
   //And add duplicate item value in arraylist.
   foreach (DataRow drow in dTable.Rows)
   {
      if (hTable.Contains(drow[colName]))
         duplicateList.Add(drow);
      else
         hTable.Add(drow[colName], string.Empty); 
   }

   //Removing a list of duplicate items from datatable.
   foreach (DataRow dRow in duplicateList)
      dTable.Rows.Remove(dRow);

   //Datatable which contains unique records will be return as output.
      return dTable;
}

Here Links below

http://www.dotnetspider.com/resources/4535-Remove-duplicate-records-from-table.aspx

http://www.dotnetspark.com/kb/94-remove-duplicate-rows-value-from-datatable.aspx

For remove duplicates in column

http://dotnetguts.blogspot.com/2007/02/removing-duplicate-records-from.html

Answer 6

Completely distinct rows:

public static DataTable Dictinct(this dt) => dt.DefaultView.ToTable(true);

Distinct by particular row(s) (Note that the columns mentioned in "distinctCulumnNames" will be returned in resulting DataTable):

public static DataTable Dictinct(this dt, params string[] distinctColumnNames) => 
dt.DefaultView.ToTable(true, distinctColumnNames);

Distinct by particular column (preserves all columns in given DataTable):

public static void Distinct(this DataTable dataTable, string distinctColumnName)
{
    var distinctResult = new DataTable();
    distinctResult.Merge(
                     .GroupBy(row => row.Field<object>(distinctColumnName))
                     .Select(group => group.First())
                     .CopyToDataTable()
            );

    if (distinctResult.DefaultView.Count < dataTable.DefaultView.Count)
    {
        dataTable.Clear();
        dataTable.Merge(distinctResult);
        dataTable.AcceptChanges();
    }
}