Writing nested JSON in spark scala

问题:

My Spark-SQL is generating an output for a query by joining two tables which has one to many cardinality. I have to convert the data into JSON. This is how the output of the query will look like.

Address_id_parent | Address_id_child | Country_child | city_child
1                 |      1           |     India     |    Delhi
1                 |      1           |     US        |    NewYork
1                 |      1           |     US        |    NewJersey

The above data has to be converted to JSON in this way.

{
    "Address": {
        "Address_id_parent": "1"
    },
    "Address-details": [{
        "Address_id_child": "1",
        "location": [{
                 "country":"India",
                 "city":"Delhi",
                },
                {
                 "country":"US",
                 "city":"NewYork",
                },
                {
                 "country":"US",
                 "city":"NewJersey",
                }
                ]
    }]
}

How can I accomplish this?

回答1:

Check Dataframe write interface with json:

df.write.format("json").save(path)