Extract English words from a text in R

Question

I have a text and I need to extract all English words from it. For instance I want to have a function which would analyse the vector

vector <- c("picture", "carpet", "lamp", "notaword", "anothernotaword")

And return only English words from this vector i.e. "picture", "carpet", "lamp"

I do understand that the definition of "English word" depends on the dictionary but I would be satisfied even with a basic dictionary.

Answer 1

You could use the package I maintain qdapDictionaries (no need for the parent package qdap to be installed). If your data is more complex you may need to use tools like tolower etc. to make it work. The idea here is basically to see where a known word list ?GradyAugmented intersects with your words. Here are two very similar approaches, the first is likely slightly faster depending on data:

vector <- c("picture", "carpet", "lamp", "notaword", "anothernotaword")

library(qdapDictionaries)
vector[vector %in% GradyAugmented]

## [1] "picture" "carpet"  "lamp"

intersect(vector, GradyAugmented)

## [1] "picture" "carpet"  "lamp"   

The error you are receiving with installing qdap sounds like @Ben Bolker is correct. You will need a newer version (I'd suggest the latest version) of data.table installed (use packageVersion("data.table") to check this). That is an oversight on my part with not requiring a minimal version of data.table, I thought setDT (a function in the data.table package) was always around but it appears to not be in your version. But to solve this particular problem you wouldn't need to install the parent qdap package, just qdapDictionaries.