Prevent unlist to drop NULL values

Question

I have a vector of lists and I use unlist on them. Some of the elements in the vectors are NULL and unlist seems to be dropping them.

How can I prevent this?

Here's a simple (non) working example showing this unwanted feature of unlist

a = c(list("p1"=2, "p2"=5), 
      list("p1"=3, "p2"=4), 
      list("p1"=NULL, "p2"=NULL), 
      list("p1"=4, "p2"=5))
unlist(a)
 p1 p2 p1 p2 p1 p2 
 2  5  3  4  4  5 

Answer 1

The issue here is that you can't have NULL in the middle of a vector. For example:

> c(1,NULL,3)
[1] 1 3

You can have NA in the middle though. You could could convert it to character and then back to numeric, which automatically converts the NULL values to NA (with a warning):

> b <- as.numeric(as.character(a))
Warning message:
NAs introduced by coercion 

then put the names back in, because they've been dropped by the previous operation:

> names(b) <- names(a)
> b
p1 p2 p1 p2 p1 p2 p1 p2 
2  5  3  4 NA NA  4  5 `

Answer 2

In this case (one level depth list) this should works too:

a[sapply(a, is.null)] <- NA
unlist(a)
# p1 p2 p1 p2 p1 p2 p1 p2 
#  2  5  3  4 NA NA  4  5

Answer 3

If you are dealing with a long complex JSON with several levels you should give this a try:

I extracted game log data from nba.com/stats web site. The problem is, some players have a NULL value for 3 point free throws (mostly centers) and jsonlite::fromJSON seems to handle NULL values very well:

#### Player game logs URL: one record per player per game played ####
gameLogsURL <- paste("http://stats.nba.com/stats/leaguegamelog?Counter=1000&Direction=DESC&LeagueID=00&PlayerOrTeam=P&Season=2016-17&SeasonType=Regular+Season&Sorter=PTS")

#### Import game logs data from JSON ####
# use jsonlite::fromJSON to handle NULL values
gameLogsData <- jsonlite::fromJSON(gameLogsURL, simplifyDataFrame = TRUE)
# Save into a data frame and add column names
gameLogs <- data.frame(gameLogsData$resultSets$rowSet)
colnames(gameLogs) <- gameLogsData$resultSets$headers[[1]]

Answer 4

The correct way to indicate a missing value is NA (not NULL). Here is another version that is working:

   a = c(list("p1"=2, "p2"=5),
      list("p1"=3, "p2"=4),
      list("p1"=NA, "p2"=NA),
      list("p1"=4, "p2"=5))
  unlist(a)

p1 p2 p1 p2 p1 p2 p1 p2 
 2  5  3  4 NA NA  4  5