encoding a JSON expression from R with jsonlite or

Question

The API I'm using needs me to send it a JSON message like:

y <- '[{"a":1, "b":"select", "c":["A", "B", "C"]}, 
       {"a":2, "b":"text"},
       {"a":3, "b":"select", "c":["D", "E", "F", "G"]}]'

Notice that in the case "b" : "select" there is another variable c with the value being an array.

When I try running fromJSON(y) (in the jsonlite package) I get a data.frame where the 3rd column is a list:

> z <- fromJSON(y) 
> class(z)
[1] "data.frame"
> z %>% glimpse()
Observations: 3
Variables: 3
$ a (int) 1, 2, 3
$ b (chr) "select", "text", "select"
$ c (list) A, B, C, NULL, D, E, F, G

But when I convert it back to JSON (which is the format I want to send to the API), the c variable appears where it shouldn't be.

[{"a":1,"b":"select","c":["A","B","C"]},
 {"a":2,"b":"text","c":{}},
 {"a":3,"b":"select","c":["D","E","F","G"]}]

Is there another R data structure that could reliably get me from a data frame to the original JSON message using jsonlite (or something else)? How would it work?

Answer 1

If you need to have a data frame to calculate things, you could do (the rmNullObs function is from here):

z <- fromJSON(y)

is.NullOb <- function(x) is.null(x) | all(sapply(x, is.null))

## Recursively step down into list, removing all such objects 
rmNullObs <- function(x) {
  x <- Filter(Negate(is.NullOb), x)
  lapply(x, function(x) if (is.list(x)) rmNullObs(x) else x)
}    

z_list <- rmNullObs(as.list(as.data.frame(t(z))))
toJSON(unname(z_list), auto_unbox=T)

This converts the dataframe to a list, removes the NULL, and converts back to JSON.

If you don't need the dataframe, see @skwon's answer

Answer 2

with jsonlite, you can do as follows:

z <- fromJSON(y, simplifyDataFrame = F)
toJSON(z, auto_unbox=T)

In addition, consider following case where last c element is ["D"]:

y2 <- '[{"a":1, "b":"select", "c":["A", "B", "C"]}, 
        {"a":2, "b":"text"}, 
        {"a":3, "b":"select", "c":["D"]}]'

z2 <- fromJSON(y2, simplifyDataFrame = F)
toJSON(z2, auto_unbox=T)

Result is this:

[{"a":1,"b":"select","c":["A","B","C"]},
 {"a":2,"b":"text"},
 {"a":3,"b":"select","c":"D"}]

This can be a trouble because the last c element is "D" not ["D"].

To prevent this, don's use auto_unbox. Use unbox carefully as follows:

z2 <- fromJSON(y2, simplifyDataFrame = F)
for(i in 1:length(z2)){
  z2[[i]][[1]] <- unbox(z2[[i]][[1]])
  z2[[i]][[2]] <- unbox(z2[[i]][[2]])
}
toJSON(z2)