Rasa: Allow extracted entities to impact intent classification

[xliuhw]

Hi, as far as I know RASA trains entity and intent classification separately (Thread #503 also mentioned it)
We found in a lot of cases they are related each other. Some entities are intent-dependent. e.g.

1. in music_play intent, artist_name entity would be confused by person_name entity in calendar_query intent

2. in email_query intent "is there any email from emilia", emilia is recognised as place_name instead of "person_name", it is more often there would be person_name than place_name in email query.

3. in taxi_booking intent "i need a taxi to pick me up at the house", instead of "place_name" the "house" is mis-recognised as "house_place" (e.g. house, living room, kitchen etc) which is defined in the intent of home scenario.
   Normally we get much higher accuracy for intent classification than the entity extraction. So I think it should improve the entity recognition if it considers the classified intent as a feature for the recognition.

I see the thread #555 talking about it. I am wondering do you have any plan for this type of entity-intent dependency considerations? Or is there any suggestions to quickly try to see the improvements?

Thanks!

[wrathagom]

@twerkmeister @amn41 not trying to beat up on #773, but this seems like another potential use case for the CRF tags you all were discussing.

[twerkmeister]

I don't know exactly how true joint training could work. Sequentially you could either use the intent as an input to the crf or extracted entities as input to the intent classifier. But then only one can benefit. #phdneeded :D

[tmbo]

Ok current idea is to let the entity extraction influence the intent classification.

[jamesmf]

I attempted to do this by modifying the tensorflow_embedding_intent_classifier to have a separate input that took intent classes, embedded them, then merged that input to the last dense layer of the text_features based input. That worked fine during training, but performance dropped off at validation time, fairly unsurprisingly.

I was using the actual tagged entities on training_data to train the intent classifier, but what I really want is to use the extracted entities from the NER. Is there a way to get that at training time?

For reference, I'm using ner_crf prior to the intent classifier.

If there is a way to do that, it wouldn't be bad to add a simple intent_featurizer_for_entity_classifier and entity_featurizer_for_intent_classifier class that (as a first pass) just adds those features straight into text_features

[tmbo]

To get the extracted entities after training the CRF, you should pass the training data through the trained classifier after the training step in the train function of the CRF. The detected entities can be added to the message object of the training examples. If the TF component is put after this one in the pipeline, it will be able to access that data.

I think we are going to integrate something like this, as soon as there is proof that it helps the classification.

[jamesmf]

I think it'd be a nice feature. The simple demonstration that it could be useful is one where you have off-the-shelf NER (spacy, say) and not much data. Explicitly passing "this example contains a name" vs trying to recognize names from the text_features of say 50 samples that contain names would be much easier.

What if every Extractor provided an add_to_extracted_entities flag? Then if it was True, you would call example.set("extracted_entities", ...) at training time.

Then an EntityFeaturizer could use extracted_entities at train time and entities for inference?

[mauricedoepke]

I have a problem where this would be very useful. I'm asking the user for his Name. ner_spacy usually extracts it right, but tensworflow has problems recognizing the intent if the user only says his name instead of 'My name is Bob'.

I will try to write a featurizer that uses the entity data to generate text features for this specific case tomorrow.

Maybe we can build a more generalized solution from this.

Has someone else uses cases that need entites influence the intent recognition? Most of the examples here are for intent classification influencing entity extraction.

[llermaly]

same issue here :)

I have this 2 examples :

"Get info(get_information) of John Doe(worker)"
"Get info (get_information) of Coca Cola(company)"

So im trying to tell Rasa :

get_information + worker entities = search_worker
get_information + company entities = search_company

Is this the way? regards

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[anil8593]

Are the tagged entity values removed while training intent engine?

[dakshvar22]

An update on this: We recently released DIETClassifier as a new component which is jointly trained to predicts intents and entities together for an input user message. You can read more about the underlying architecture and the results in our paper and this blogpost. You can also watch this 3 video series on DIET to understand how it works.