JQ - Merge two arrays

问题:

I'm looking for JQ query that allows me to merge 2 arrays variables (not files) and also take me to overwrite first array with newer value from second array. For example:

#!/bin/bash -e

firstArrayVariable='
    [
        {
            "Key": "A B",
            "Value": "1 2"
        },
        {
            "Key": "C D",
            "Value": "3 4"
        },
        {
            "Key": "E F",
            "Value": "5 6"
        },
        {
            "Key": "G H",
            "Value": "9 10"
        }
    ]
'

secondArrayVariable='
    [
        {
            "Key": "A B",
            "Value": "1 2"
        },
        {
            "Key": "C D",
            "Value": "3 4"
        },
        {
            "Key": "G H",
            "Value": "11 12"
        },
        {
            "Key": "J K",
            "Value": "15 16"
        }
    ]
'

jq \
    --compact-output \
    --raw-output \
    --arg jqSecondArrayVariable "${secondArrayVariable}" \
    '. + $jqSecondArrayVariable // empty' \
<<< "${firstArrayVariable}"

I could not get it to work and I got following error

jq: error (at :19): array ([{"Key":"A ...) and string ("\n [\n ...) cannot be added

What I expect the result of the merged array is

[
    {
        "Key": "A B",
        "Value": "1 2"
    },
    {
        "Key": "C D",
        "Value": "3 4"
    },
    {
        "Key": "E F",
        "Value": "5 6"
    },
    {
        "Key": "G H",
        "Value": "11 12"
    },
    {
        "Key": "J K",
        "Value": "15 16"
    }
]

any helps would be very appreciated!

UPDATED

I tried to use --argjson as @peak suggested, it concatenate array but it could not merge 2 arrays. Result I got now is an array with duplicated objects

回答1:

Assuming that 2 input arrays are named as firstArr and secondArr respectively.

with group_by() (to group objects by the crucial key "Key") and map() functions:

jq --argjson arr1 "$firstArr" --argjson arr2 "$secondArr" -n \
'$arr1 + $arr2 | group_by(.Key) | map(.[-1])'

The output:

[
  {
    "Key": "A B",
    "Value": "1 2"
  },
  {
    "Key": "C D",
    "Value": "3 4"
  },
  {
    "Key": "E F",
    "Value": "5 6"
  },
  {
    "Key": "G H",
    "Value": "11 12"
  },
  {
    "Key": "J K",
    "Value": "15 16"
  }
]

---

Alternatively and in such particular case, a faster way, you could apply the following trick with unique_by() function:

jq --argjson arr1 "$firstArr" --argjson arr2 "$secondArr" -n '$arr2 + $arr1 | unique_by(.Key)'

回答2:

You should be using -—argjson; —-arg interprets its argument as a JSON string. You will also have to modify your jq filter, because simply adding the arrays will result in their concatenation.

For further guidance, see e.g. Combining JSON by common key-value pairs