What is a chunker in Natural Language Processing?

Question

Does anyone know what is a chunker in the context of text processing and what is it's usage?

Answer 1

I don't personally disagree with the other answers, but Jurafsky and Martin give a slightly different definition. For them, chunking is specifically the type of shallow parsing in which there are no recursive phrases.

One example they give is the phrase "the flight from Denver". One parse that would not be generated by a chunker is "[NP the flight [PP from [NP Denver]]]" because it implies a grammar with NP-recursivity.

Answer 2

According to these slides, chunking is an alternative to parsing that provides a partial syntactic structure of a sentence, with a limited tree depth, as opposed to full on parsing.

It is more limited than full parsing, but is sufficient when it comes to extracting or ignoring information, and is thus many times used, as it's faster and more robust than parsing.

Much more information is available in the slides.

Further links:

• More slides
• Notes from lectures at NYU

Answer 3

It's a very simplistic type of parsing, called shallow parsing. The OpenNLP project has a chunker module available, and you can see its documentation for an example of chunking in action