Ordering a dataframe by its subsegments

问题:

My team and I are dealing with many thousands of URLs that have similar segments. Some URLs have one segment ("seg", plural, "segs") in a position of interest to us. Other similar URLs have a different seg in the position of interest to us. We need to sort a dataframe consisting of URLs and associated unique segs in the position of interest, showing the frequency of those unique segs.

Here is a simplified example:

 url <- c(1, 3, 1, 4, 2, 3, 1, 3, 3, 3, 3, 2)
 seg <- c("a", "c", "a", "d", "b", "c", "a", "x", "x", "y", "c", "b")
 df <- data.frame(url,seg)

We are looking for the following:

url freq seg 
 1   3    a   in other words, url #1 appears three times each with a seg = "a",
 2   2    b   in other words: url #2 appears twice each with a seg = "b",
 3   3    c   in other words: url #3 appears three times with a seg = "c", 
 3   2    x                                  two times with a seg = "x", and, 
 3   1    y                                  once with a seg = "y"
 4   1    d   etc.

I can get there using a loop and several small steps, but am convinced there is a more elegant way of doing this. Here's my inelegant approach:

Create empty dataframe with num.unique rows and three columns (url, freq, seg)

 result <- data.frame(url=0, Freq=0, seg=0)

Determine the unique URLs

 unique.df.url <- unique(df$url)

Loop through the dataframe

 for (xx in unique.df.url) {
   url.seg <- df[which(df$url == unique.df.url[xx]), ] # create a dataframe for each of the unique urls and associated segs
   freq.df.url <- data.frame(table(url.seg))  # summarize the frequency distribution of the segs by url
   result <- rbind(result,freq.df.url)  # append a new data.frame onto the last one
 }

Eliminate rows in the dataframe where Frequency = 0

 result.freq <- result[which(result$Freq |0), ]

Sort the dataframe by URL

 result.order <- result.freq[order(result.freq$url), ]

This yields the desired results, but since it is so inelegant, I am concerned that once we move to scale, the time required will be prohibitive or at least a concern. Any suggestions?

回答1:

In base R you can do this :

aggregate(freq~seg+url,`$<-`(df,freq,1),sum)
# or aggregate(freq~seg+url, data.frame(df,freq=1),sum)

#   seg url freq
# 1   a   1    3
# 2   b   2    2
# 3   c   3    3
# 4   x   3    2
# 5   y   3    1
# 6   d   4    1

The trick with $<- is just to add a column freq of value 1 everywhere, without changing your source table.

Another possibility:

subset(as.data.frame(table(df[2:1])),Freq!=0)
#    seg url Freq
# 1    a   1    3
# 8    b   2    2
# 15   c   3    3
# 17   x   3    2
# 18   y   3    1
# 22   d   4    1

Here I use [2:1] to switch the order of columns so table orders the results in the required way.

回答2:

url <- c(1, 3, 1, 4, 2, 3, 1, 3, 3, 3, 3, 2)
seg <- c("a", "c", "a", "d", "b", "c", "a", "x", "x", "y", "c", "b")
df <- data.frame(url,seg)

library(dplyr)

df %>% count(url, seg) %>% arrange(url, desc(n))

# # A tibble: 6 x 3
#     url seg       n
#   <dbl> <fct> <int>
# 1     1 a         3
# 2     2 b         2
# 3     3 c         3
# 4     3 x         2
# 5     3 y         1
# 6     4 d         1

回答3:

Would the following code be better for you?

library(dplyr)
df %>% group_by(url, seg) %>% summarise(n()) 

回答4:

Or paste & tapply:

url <- c(1, 3, 1, 4, 2, 3, 1, 3, 3, 3, 3, 2)
seg <- c("a", "c", "a", "d", "b", "c", "a", "x", "x", "y", "c", "b")
df <- data.frame(url,seg)

want <- tapply(url, INDEX = paste(url, seg, sep = "_"), length)
want <- data.frame(do.call(rbind, strsplit(names(want), "_")), want)
colnames(want) <- c("url", "seg", "freq")
want <- want[order(want$url, -want$freq), ]
rownames(want) <- NULL # needed?
want <- want[ , c("url", "freq", "seg")] # needed?
want

回答5:

An option can be to use table and tidyr::gather to get data in format needed by OP:

library(tidyverse)
table(df) %>% as.data.frame() %>% 
  filter(Freq > 0 ) %>%
  arrange(url, desc(Freq))


#   url seg  Freq
# 1   1   a     3
# 2   2   b     2
# 3   3   c     3
# 4   3   x     2
# 5   3   y     1
# 6   4   d     1

OR

df %>% group_by(url, seg) %>%
  summarise(freq = n()) %>%
  arrange(url, desc(freq))

# # A tibble: 6 x 3
# # Groups: url [4]
#    url seg      freq
#   <dbl> <fctr> <int>
# 1  1.00 a          3
# 2  2.00 b          2
# 3  3.00 c          3
# 4  3.00 x          2
# 5  3.00 y          1
# 6  4.00 d          1