How to remove duplicate values from a multi-dimens

Question

How can I remove duplicate values from a multi-dimensional array in PHP?

Example array:

Array
(
    [0] => Array
    (
        [0] => abc
        [1] => def
    )

    [1] => Array
    (
        [0] => ghi
        [1] => jkl
    )

    [2] => Array
    (
        [0] => mno
        [1] => pql
    )

    [3] => Array
    (
        [0] => abc
        [1] => def
    )

    [4] => Array
    (
        [0] => ghi
        [1] => jkl
    )

    [5] => Array
    (
        [0] => mno
        [1] => pql
    )

)

Answer 1

I've given this problem a lot of thought and have determined that the optimal solution should follow two rules.

1. For scalability, modify the array in place; no copying to a new array
2. For performance, each comparison should be made only once

With that in mind and given all of PHP's quirks, below is the solution I came up with. Unlike some of the other answers, it has the ability to remove elements based on whatever key(s) you want. The input array is expected to be numeric keys.

$count_array = count($input);
for ($i = 0; $i < $count_array; $i++) {
    if (isset($input[$i])) {
        for ($j = $i+1; $j < $count_array; $j++) {
            if (isset($input[$j])) {
                //this is where you do your comparison for dupes
                if ($input[$i]['checksum'] == $input[$j]['checksum']) {
                    unset($input[$j]);
                }
            }
        }
    }
}

The only drawback is that the keys are not in order when the iteration completes. This isn't a problem if you're subsequently using only foreach loops, but if you need to use a for loop, you can put $input = array_values($input); after the above to renumber the keys.

Answer 2

The user comments on the array_unique() documentation have many solutions to this. Here is one of them:

kenrbnsn at rbnsn dot com
27-Sep-2005 12:09

Yet another Array_Unique for multi-demensioned arrays. I've only tested this on two-demensioned arrays, but it could probably be generalized for more, or made to use recursion.

This function uses the serialize, array_unique, and unserialize functions to do the work.

function multi_unique($array) {
    foreach ($array as $k=>$na)
        $new[$k] = serialize($na);
    $uniq = array_unique($new);
    foreach($uniq as $k=>$ser)
        $new1[$k] = unserialize($ser);
    return ($new1);
}

This is from http://ca3.php.net/manual/en/function.array-unique.php#57202.

Answer 3

Lots of person asked me how to make Unique multidimensional array. I have taken reference from your comment and it helps me.

First of All, Thanks to @jeromegamez @daveilers for your solution. But every time i gave the answer, they asked me how this 'serialize' and 'unserialize' works. That's why i want to share the reason of this with you so that it will help more people to understand the concept behind this.

I am explaining why we use 'serialize' and 'unserialize' in steps :

Step 1: Convert the multidimensional array to one-dimensional array

To convert the multidimensional array to a one-dimensional array, first generate byte stream representation of all the elements (including nested arrays) inside the array. serialize() function can generate byte stream representation of a value. To generate byte stream representation of all the elements, call serialize() function inside array_map() function as a callback function. The result will be a one dimensional array no matter how many levels the multidimensional array has.

Step 2: Make the values unique

To make this one dimensional array unique, use array_unique() function.

Step 3: Revert it to the multidimensional array

Though the array is now unique, the values looks like byte stream representation. To revert it back to the multidimensional array, use unserialize() function.

$input = array_map("unserialize", array_unique(array_map("serialize", $input)));

Thanks again for all this.

Answer 4

A very easy and logical way to Unique a multi dimension array is as follows,

If you have array like this:

Array
(
    [Key1] => Array
        (
            [0] => Value1
            [1] => Value2
            [2] => Value1
            [3] => Value3
            [4] => Value1
        )
    [Key2] => Array
        (
            [0] => Value1
            [1] => Value2
            [2] => Value1
            [3] => Value3
            [4] => Value4
        )
)

use foreach to solve this:

foreach($array as $k=>$v){
    $unique=array_unique($v);
    $array[$k]=$unique;
}

it will give you following result:

Array
(
    [Key1] => Array
        (
            [0] => Value1
            [1] => Value2
            [3] => Value3
        )
    [Key2] => Array
        (
            [0] => Value1
            [1] => Value2
            [3] => Value3
            [4] => Value4
        )
)

and if you want to rearrange the order of the keys,

foreach($array as $k=>$v){
    $unique= array_values(array_unique($v));
    $array[$k]=$unique;
}

This operation will give you arranged key values like this:

Array
(
    [Key1] => Array
        (
            [0] => Value1
            [1] => Value2
            [2] => Value3
        )
    [Key2] => Array
        (
            [0] => Value1
            [1] => Value2
            [2] => Value3
            [3] => Value4
        )
)

I hope this will clear everything.

Answer 5

An alternative to serialize and unique

$test = [
    ['abc','def'],
    ['ghi','jkl'],
    ['mno','pql'],
    ['abc','def'],
    ['ghi','jkl'],
    ['mno','pql'],
];

$result = array_reduce(
    $test,
    function($carry,$item){
        if(!in_array($item,$carry)) {
            array_push($carry,$item);
        }
        return $carry;
    },
    []
);

var_dump($result);

/*
 php unique.php
array(3) {
    [0] =>
        array(2) {
            [0] =>
                string(3) "abc"
            [1] =>
                string(3) "def"
        }
    [1] =>
        array(2) {
            [0] =>
                string(3) "ghi"
            [1] =>
                string(3) "jkl"
        }
    [2] =>
        array(2) {
              [0] =>
                  string(3) "mno"
              [1] =>
                  string(3) "pql"
        }
}

*/

Answer 6

If you have an array like this

data = array
(
[0] => array
(
    [subject] => a
    [object] => c
),
[1] => array
(
    [subject] => b
    [object] => d
),
[2] => array
(
    [subject] => d
    [object] => b
),
[3] => array
(
    [subject] => d
    [object] => c
),
[4] => array
(
    [subject] => c
    [object] => a
),
[5] => array
(
    [subject] => c
    [object] => d
)
)

and you want to get arrays like this:

data = array
(
[0] => array
(
    [subject] => a
    [object] => c
),
[1] => array
(
    [subject] => b
    [object] => d
),
[2] => array
(
    [subject] => d
    [object] => c
)
)

or

data = array
(
[0] => array
(
    [subject] => d
    [object] => b
),
[1] => array
(
    [subject] => c
    [object] => a
),
[2] => array
(
    [subject] => c
    [object] => d
)
)

a following code can help

    $data1 = array();
    $data1 = $data;
    for($q=0;$q<count($data);$q++)
    {
            for($p=0;$p<count($data1);$p++)
            {
                    if (($data[$q]["subject"] == $data1[$p]["object"]) && ($data[$q]["object"] == $data1[$p]["subject"]))
                    {
                            $data1[$p]["subject"] = $data[$q]["subject"];
                            $data1[$p]["object"] = $data[$q]["object"];
                    }
            }
    }
    $data1 = array_values(array_map("unserialize", array_unique(array_map("serialize", $data1))));
    $data = $data1;