how to merge two JSON objects using jq?

Question

I have two file json. I want to append two array of SomeFile2.json to SomeFile1.json as below.

SomeFile1.json

[
  {
    "DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0886ed703de64028a"
      }
    ]
  },
  {
    "DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0561634c4g3b4fa25"
      }
    ]
  }
]

SomeFile2.json

[
  {
    "InstanceId": "i-0886ed703de64028a",
    "State": "InService"
  },
  {
    "InstanceId": "i-0561634c4g3b4fa25",
    "State": "InService"
  }
]

I want the result as below:

[
  {
    "DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0886ed703de64028a"
        "State": "InService"
      }
    ]
  },
  {
    "DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0561634c4g3b4fa25"
        "State": "InService"
      }
    ]
  }
]

I'm processing in bash shell via jq. But, unsuccessful.

Answer 1

Since I find jq pretty hard, I started in a procedural way: using ruby's json module:

ruby -rjson -e '
  states = JSON.parse(File.read(ARGV.shift)).map {|o| [o["InstanceId"], o["State"]]}.to_h
  data = JSON.parse(File.read(ARGV.shift))
  data.each do |obj|
    obj["Instances"].each do |instance|
      instance["State"] = states[instance["InstanceId"]] || "unknown"
    end
  end
  puts JSON.pretty_generate data
' SomeFile2.json SomeFile1.json 

But we want jq, so after some trial and error, and finding this in the manual: https://stedolan.github.io/jq/manual/#Complexassignments -- (note, I changed the state for one of the instances so I could verify the output better)

$ cat SomeFile2.json 
[
  {
    "InstanceId": "i-0886ed703de64028a",
    "State": "InService"
  },
  {
    "InstanceId": "i-0561634c4g3b4fa25",
    "State": "NOTInService"
  }
]

First, extract the states into an object mapping the id to the state:

$ state_map=$( jq -c 'map({"key":.InstanceId, "value":.State}) | from_entries' SomeFile2.json )
$ echo "$state_map"
{"i-0886ed703de64028a":"InService","i-0561634c4g3b4fa25":"NOTInService"}

Then, update the instances in the first file:

jq --argjson states "$state_map" '.[].Instances[] |= . + {"State": ($states[.InstanceId] // "unknown")}' SomeFile1.json

[
  {
    "DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0886ed703de64028a",
        "State": "InService"
      }
    ]
  },
  {
    "DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
    "Instances": [
      {
        "InstanceId": "i-0561634c4g3b4fa25",
        "State": "NOTInService"
      }
    ]
  }
]

Answer 2

Since the contents of the second file are evidently intended to define a mapping from InstanceId to State, let's start by hypothesizing the following invocation of jq:

jq --argfile dict SomeFile2.json -f program.jq SomeFile1.json

Next, let's create a suitable dictionary:

reduce $dict[] as $x ({}; . + ($x|{(.InstanceId): .State}))) as $d

Now the rest is easy:

map(.Instances |= map(. + {State: $d[.InstanceId]}))

Putting the pieces together in program.jq:

(reduce $dict[] as $x ({}; . + ($x|{(.InstanceId): .State}))) as $d
| map(.Instances |= map(. + {State: $d[.InstanceId]}))

Alternatives

The dictionary as above can be constructed without using reduce, as follows:

($dict | map( {(.InstanceId): .State}) | add) as $d

Another alternative is to use INDEX/2:

(INDEX($dict[]; .InstanceId) | map_values(.State))) as $d

If your jq does not have INDEX/2 you can snarf its def from https://raw.githubusercontent.com/stedolan/jq/master/src/builtin.jq