I have a JSON file and the keys would be my column while loading into Spark SQL. Now when i want to retrieve the column names, it was retrieved in Alphabetical order. But i want the details should be in the order of how its present in the file
my input data is
{"id":1,"name":"Judith","email":"jknight0@google.co.uk","city":"Évry","country":"France","ip":"199.63.123.157"}
Below is my way to retrieve the column names and build a single string
val dataframe = sqlContext.read.json("/virtual/home/587635/users.json")
val columns = dataframe.columns
var query = columns.apply(0)+" STRING"
for (a <- 1 to (columns.length-1))
{
query = query + ","+ columns.apply(a) + " STRING"
}
println(query)
This gives me the output like below
city STRING,country STRING,email STRING,id STRING,ip STRING,name STRING
But i want my output as
id STRING,name STRING,email STRING,city STRING,country STRING,ip STRING
Add a
select
with the columns correctly orderedIf you tried this at the shell, you will notice the correct order
By executing the rest of your script, the output is as expected