How to remove duplicates from a list using an auxi

Question

I am trying to remove duplicates from a list by creating a temporary array that stores the indices of where the duplicates are, and then copies off the original array into another temporary array while comparing the indices to the indices I have stored in my first temporary array.

public void removeDuplicates()
{
    double tempa [] = new double [items.length];
    int counter = 0;
    for ( int i = 0; i< numItems ; i++)
    {
        for(int j = i + 1; j < numItems; j++)
        {
            if(items[i] ==items[j])
            {
                tempa[counter] = j;
                counter++;

            }
        }
    }

    double tempb [] = new double [ items.length];
    int counter2 = 0;
    int j =0;
    for(int i = 0; i < numItems; i++)
    {
        if(i != tempa[j])
        {
            tempb[counter2] = items[i];
            counter2++;

        }
        else
        {
            j++;

        }
    }

    items = tempb;
    numItems = counter2;
}

and while the logic seems right, my compiler is giving me an arrayindexoutofbounds error at

tempa[counter] = j;

I don't understand how counter could grow to above the value of items.length, where is the logic flaw?

Answer 1

Done for int arrays, but easily coud be converted to double.

1) If you do not care about initial array elements order:

private static int[] withoutDuplicates(int[] a) {
    Arrays.sort(a);
    int hi = a.length - 1;
    int[] result = new int[a.length];
    int j = 0;
    for (int i = 0; i < hi; i++) {
        if (a[i] == a[i+1]) {
            continue;
        }
        result[j] = a[i];
        j++;            
    }
    result[j++] = a[hi];
    return Arrays.copyOf(result, j);
}

2) if you care about initial array elements order:

private static int[] withoutDuplicates2(int[] a) {
    HashSet<Integer> keys = new HashSet<Integer>();
    int[] result = new int[a.length];
    int j = 0;
    for (int i = 0 ; i < a.length; i++) {
        if (keys.add(a[i])) {
            result[j] = a[i];
            j++;
        }
    }
    return Arrays.copyOf(result, j);
}

3) If you do not care about initial array elements order:

private static Object[] withoutDuplicates3(int[] a) {
    HashSet<Integer> keys = new HashSet<Integer>();
    for (int value : a) {
        keys.add(value);
    }
    return keys.toArray();
}

Answer 2

Imagine this was your input data:

Index: 0, 1, 2, 3, 4, 5, 6, 7, 8
Value: 1, 2, 3, 3, 3, 3, 3, 3, 3

Then according to your algorithm, tempa would need to be:

Index: 0, 1, 2, 3, 4, 5, 6, 7, 8, ....Exception!!!
Value: 3, 4, 5, 6, 7, 8, 4, 5, 6, 7, 8, 5, 6, 7, 8, 6, 7, 8, 7, 8, 8

Why do you have this problem? Because the first set of nested for loops does nothing to prevent you from trying to insert duplicates of the duplicate array indices!

What is the best solution?

Use a Set! Sets guarantee that there are no duplicate entries in them. If you create a new Set and then add all of your array items to it, the Set will prune the duplicates. Then it is just a matter of going back from the Set to an array.

Alternatively, here is a very C-way of doing the same thing:

//duplicates will be a truth table indicating which indices are duplicates.
//initially all values are set to false
boolean duplicates[] = new boolean[items.length];
for ( int i = 0; i< numItems ; i++) {
    if (!duplicates[i]) { //if i is not a known duplicate
        for(int j = i + 1; j < numItems; j++) {
            if(items[i] ==items[j]) {
                duplicates[j] = true; //mark j as a known duplicate
            }
        }
    }
}

I leave it to you to figure out how to finish.