tm loses the metadata when applying tm_map

2019-03-31 17:53发布

I have a (small) problem with the tm r library. say I have a corpus:

# boilerplate
bcorp <- c("one","two","three","four","five")
myCorpus <- Corpus(VectorSource(bcorp), list(lanuage = "en_US"))
tdm <- TermDocumentMatrix(myCorpus)
Docs(tdm)

Result:

[1] "1" "2" "3" "4" "5"

This works. But when I try to use a transformation tm_map():

# this does not work
myCorpus <- Corpus(VectorSource(bcorp), list(lanuage = "en_US"))
myCorpus <- tm_map(myCorpus, tolower)
tdm <- TermDocumentMatrix(myCorpus)

Gives

Error: inherits(doc, "TextDocument") is not TRUE

The solution proposed in this case was to transform to PlainTextDocument.

# this works but erase the metadata
myCorpus <- Corpus(VectorSource(bcorp), list(lanuage = "en_US"))
myCorpus <- tm_map(myCorpus, tolower)
myCorpus <- tm_map(myCorpus, PlainTextDocument)
tdm <- TermDocumentMatrix(myCorpus)
Docs(tdm)

Result:

[1] "character(0)" "character(0)" "character(0)" "character(0)" "character(0)"

Now it works, but erase all the metadata (in this case the doc names). There is a way to mantain the metadata, or to save and then restore them?

标签: r metadata tm
1条回答
疯言疯语
2楼-- · 2019-03-31 18:18

I found it.

The line:

myCorpus <- tm_map(myCorpus, PlainTextDocument)

solves the problem but erase the metadata.

I found this answer that explain a better way to use tm_map(). I just have to substitute:

myCorpus <- tm_map(myCorpus, tolower)

with:

myCorpus <- tm_map(myCorpus, content_transformer(tolower))

And all works!

查看更多
登录 后发表回答