How rasa_nlu using lookup_tables for entity extrac

Question

I am trying to develop a chatbot using rasa nlu and rasa core. But I am not getting the link how rasa_nlu using lookup_tables for entity extraction. I had already go through (http://blog.rasa.com/improving-entity-extraction/) link but not getting it. How should lookup_table used for to extract the entity?

Answer 1

Requirements:

If you want to use lookup tables, make sure:

• you have the components intent_entity_featurizer_regex and ner_crf in your NLU pipeline
• the entities you want to match fit have a well defined and narrow scope
  • entities like food names, company names, car brands are unlikely to appear in contexts you in which you don't want to match them. Hence, look up tables are a good use case for them.
  • entities like objects (e.g. "car", "house", "paper") appear in a variety of contexts in which you don't want to match them at all. Therefore, using look up tables might even lead to worse results.

In your training data

In order to use look up tables, you can either define them directly in the training data, e.g.:

## intent:check_balance
- what is my balance <!-- no entity -->
- Could I pay in [yen](currency)?  <!-- entity matched by lookup table -->

## lookup:currency   <!-- lookup table list -->
- Yen
- USD
- Euro

Or you can write them in a text file:

Yen
USD
Euro

And then include the path to the text file in your training data:

## intent:check_balance
   ... like before

## lookup:food
    <path to your look up table text file>.txt

Taking an input like Could I pay in Euro?, Rasa NLU then sets the value of the slot currency to Euro.

How they work

The single items in a look up table are added to a regular expression (regex) which is applied to the the messages which your users send to the bot. However, look up tables don't work if your user inserts typos, e.g. a look up table entry Pesos would not match Peesos. To also match these cases you can try fuzzy matching which is described in the blog article you linked. Make sure that your look up tables don't become too large as Rasa NLU has to check every sentence whether it matches one of your look up table entries.

Maybe the Rasa NLU documentation can also help you.