问题:

Does anyone know what is a chunker in the context of text processing and what is it's usage?

回答1:

According to these slides, chunking is an alternative to parsing that provides a partial syntactic structure of a sentence, with a limited tree depth, as opposed to full on parsing.

It is more limited than full parsing, but is sufficient when it comes to extracting or ignoring information, and is thus many times used, as it's faster and more robust than parsing.

Much more information is available in the slides.

Further links:

More slides
Notes from lectures at NYU

回答2:

I don't personally disagree with the other answers, but Jurafsky and Martin give a slightly different definition. For them, chunking is specifically the type of shallow parsing in which there are no recursive phrases.

One example they give is the phrase "the flight from Denver". One parse that would not be generated by a chunker is "[NP the flight [PP from [NP Denver]]]" because it implies a grammar with NP-recursivity.

回答3:

It's a very simplistic type of parsing, called shallow parsing. The OpenNLP project has a chunker module available, and you can see its documentation for an example of chunking in action