I currently have a dataframe with 260,000 rows and 50 columns where 3 columns are numeric and the rest are categorical. I wanted to one hot encode the categorical columns in order to perform PCA and use regression to predict the class. How can I go about accomplishing the below example in R?
Example:
V1 V2 V3 V4 V5 .... VN-1 VN
to
V1_a V1_b V2_a V2_b V2_c V3_a V3_b and so on
You can use model.matrix
or sparse.model.matrix
. Something like this:
sparse.model.matrix(~. -1, data = your_data)
The ~.
tells R that your entire table (the .
) is the right hand side of some hypothetical model, and the -1
says to leave out the intercept. Without the -1
your first column will be a vector of 1s.
Don't really what you mean by "hot encode".
Here's an example of using dplyr to spread out the catagorical variable iris$Species into three separate columns:
df <- iris %>%
mutate(id = rownames(.) %>% # unique identified to prevent duplicate rows when spreading
mutate(val=1) %>% # give the categorical variable a value of 1
spread(Species, val) # spread out each level of iris$Species as columns
df[76:80,]
Sepal.Length Sepal.Width Petal.Length Petal.Width id setosa versicolor virginica
76 5.8 2.7 4.1 1.0 68 NA 1 NA
77 5.8 2.7 5.1 1.9 102 NA NA 1
78 5.8 2.7 5.1 1.9 143 NA NA 1
79 5.8 2.8 5.1 2.4 115 NA NA 1
80 5.8 4.0 1.2 0.2 15 1 NA NA
Basically a oneliner with data.table
and mltools
:
# data.table with 125 variables:
dt_1h <- one_hot(dt)
# MD5 for checking reproducibility:
> digest::digest(dt_1h, algo = "md5")
[1] "f1eb1c1e2d5d94b709101557c9ed8d0d"
Data
library(data.table)
library(mltools)
set.seed(1701)
df <- data.frame(matrix(sample(c(LETTERS[1:26]),
260000*3, replace = TRUE), ncol = 3),
matrix(rnorm(260000*47), ncol = 47))
dt <- as.data.table(df)