A follow-up to “Extracting data from an API using

Question

The code I have (which comes from here A continuation of... Extracting data from an API using R) gives a very complicated output. I can extract almost all I need except for a data.frame that's nested within the list.

Without doing anything, it gives me this error:

Error in .rowNamesDF<-(x, value = value) : duplicate 'row.names' are not allowed In addition: Warning message: non-unique values when setting 'row.names': ‘1’, ‘10’, ‘11’, ‘12’, ‘13’, ‘14’, ‘15’, ‘16’, ‘17’, ‘18’, ‘19’, ‘2’, ‘20’, ‘3’, ‘4’, ‘5’, ‘6’, ‘7’, ‘8’, ‘9’

If I try to flatten or unlist, it comes up NULL.

In the example code, I've added some variables that are easy to get and number 42 is "dokintressent", from which I need "intressent", a list of names for each case. I have to run APIs from the Swedish legislative a half a dozen times, but this is the trickier one.

When I remove 42, it makes the data.frame perfectly.

my_dfs1 <- lapply(1:207, function(i){
  my_url <- paste0("http://data.riksdagen.se/dokumentlista/?sok=&doktyp=mot&rm=&from=2017-01-01&tom=2017-12-31&ts=&bet=&tempbet=&nr=&org=&iid=&webbtv=&talare=&exakt=&planering=&sort=rel&sortorder=desc&rapport=&utformat=json&a=s&p=", i)
  r1 <- GET(my_url)
  r2 <- rawToChar(r1$content)
  r3 <- fromJSON(r2)
  r4 <- r3$dokumentlista$dokument
  return(r4)
})

df <- my_dfs1 %>% lapply(function(df_0){
  df_0[c(12:14, 18, 42)]
}) %>% do.call(rbind, .)

EDIT: I've noticed that the data I want is actually several data.frames per case. From "intressent", I need "namn". Basically, I need the final database to look like this:

                     V12     V13    V14    V18    Namn
    Motion 1                                     c(name1, name2)

Answer 1

you need to work on intressent on its own and extract from it what you need and then assign it to a new column, just make sure you get a simple data structure per row.

You can also, if it works better for you, paste the names together, separated by '-', for example, and then intressent will be a simple character vector.

df <- my_dfs1 %>% lapply(function(df_0){
  #choose the columns you want
  return_df <- df_0[c(12:14, 18)]
  # work on intressent
  return_df$namn <- df_0$dokintressent$intressent %>% 
    lapply(function(x)list(x$namn)) %>% 
    do.call(rbind, .)                    # careful here a simple unlist won't work
  return(return_df) }) %>% 
  do.call(rbind, .)