Botframework-sdk: Multiple intents on just one utterance (LUIS)

[piffarettig]

Just that, is it possibly to detect two intents on just one utterance using LUIS?

Ex: 'I want to do A and then B'

Where LUIS should respond me with both A and B as intents.

Every utterance that contains an 'and' has two intents (or more).

Thanks!

[piffarettig]

I was thinking of defining two intents on LUIS, and add them (apart from their required entities), another entity which would be 'secondIntent'. That entity just represents another intent, where that entity should always start with 'and' followed with a text that is the intent text. Then, by code, on my bot, whenever the first intent get called and a secondIntent entity is detected, I call LUIS again manually with that chunk of text (the one that LUIS in first place associated to the intent as a 'secondIntent' entity).

This is not the cleanest solution and would imply to make some recursion if the secondIntent has more than just one 'and'. In addition, it will also mean to add that possible 'secondIntent' to every single intent in my Luis Model.

So... any ideas to make this better?

[GanadiniAkshay]

A simple approach that I used is to split sentences based on simple keywords such as and etc or use NLTK for complicated cases. And then make a parallel async request to LUIS for all the separate parts of the sentence.

This helps me keep my LUIS models clean and semantically correct.

[piffarettig]

Cool, thanks for your answer. I still have some doubts:

1) How do you call NLTK from C# code? Do you got any example?
2) And where do you make the parallel requests to LUIS model again?

Currently I am using LUIS with a LuisDialog, so I have a separate method for every intent. So, for instance what you say is to do sth like this:

[LuisIntent("SomeIntentThatCanContainMultipleInnerIntents")] public async Task ProcessMultipleIntents(IDialogContext context, LuisResult result) { IntentRecommendation intentReceived = result.Intents[0]; if (IntentContainsManyInnerIntents(intentReceived)) //this would use NLTK { List<string> innerIntents = GetAllInnerIntents(intentReceived); //this would use NLTK foreach (string intent in innerIntents) { CallLUIS(intent); } } else { DoOtherStuff(); } }

Correct me if I got you wrong. An example of what you usually do could really help!

Thanks in advance @GanadiniAkshay

[GanadiniAkshay]

@piffarettig I actually did this in node.js but I will try making an example in C# over the weekend

[piffarettig]

@GanadiniAkshay Ok. WIll be waiting for your answer! Thanks!

[GanadiniAkshay]

@piffarettig sorry been really with something (making api.ai recognizer for the bot framework...will look into this today or tomorrow) sorry for the delay

[chrimc62]

And just to confirm LUIS does not allow recognizing multiple intents in one sentence. Depending on your needs you can sometimes work around this by using entities instead which you can have multiple in one sentence.

[piffarettig]

@chrimc62 thanks! yes, thats the workaround I am using right know, but as I said, I'm kind of polluting my intents with entities and always doing some kind of recursion calling Luis until no more intent is found in the sentence. Probably @GanadiniAkshay approach is the cleanest for this, as it doesn't add complexity to the Luis model.

[piffarettig]

@GanadiniAkshay could you post the node code ? I could adapt it to c#. Thanks!

[chrimc62]

@GanadiniAkshay can you help?

[GanadiniAkshay]

Sorry @piffarettig @chrimc62 been really busy with a few things.

Here is the high level way of solving this...its not perfect but better than the original OP solution of entities..

First step is to divide the text into sentences based on key words and punctuations

c# code

//split on punctuation
string[] sentences = Regex.Split(input, @"(?<=[.!\?])\s+");

//split on keywords
string[] sentences = input.split('and'); //do similarly for other keywords

And once you have the sentences you will have to make api calls to LUIS

https://dev.projectoxford.ai/docs/services/56d95961e597ed0f04b76e58/operations/56f8a55119845511c81de488

I realized that this is actaully quite complicated and one more problem is you won't be able to use the inbuilt IDialog or LuisDialog class. You will either have to modify or use your own handlers.

So its really a trade off between clean way vs easy way

[piffarettig]

These sentences you are getting doesn't always mean separate intents. There are lots of examples where you could use punctuation symbols like dots or commas, or separator words like 'and' and 'or', and be expressing only one intent.

For instance:

_'I want a large pizza, with pepperonis and blue cheese.'
'Book a room for two with jacuzzi or at least with a buthtab, please'_

I followed a similar approach but instead of doing it all by hand on C# code I trained my LUIS model to detect these 'smaller sentences' that refer to other intents inside sentences (or utterances).

So yeah, I followed my previous idea of using an extra entity per LUIS model. By doing this delegate the model all the responsibility of parsing an intent and its possible inner intents. And I am using a LuisDialog for doing this. Whenever the appropiate intent handler gets called, some logic is done according to that intent, and that extra entity is read.

In case that entity contains a value, I modify the message (replacing the original text with the one contained in the entity), and I re dispatch it to the LuisDialog. Being it called again. This will be done with each smaller and smaller sentence until the entity that refers to an extra intent has an empty value.

I think that's purpose of LUIS, it has lots of heavy machine learning algorithms behind, that are proved and allow you as a developer to use and train a model really easily. The output you re going to get from it as a 'something that understand language' is going to be richer than some simple regular expressions that are static in some way.

In my opinion, doing a trade-off balance between both options, understanding the intents using LUIS is more accurate, and the only disadvantage is to add a simple extra entity and train the model to match it.

[Owen864720655]

Hi all. I am also working on recognizing multiple intents from one single sentence. Are there any general approaches or papers by which I can have some ideas on this thing?

[SharmaAjay19]

You can always use the verbose=true query parameter when you hit your prediction request. It throws the scores for all intents and then you can shortlist the top two or three.
Check sample here: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-concept-data-extraction

[michaelkuzmin]

You can always use the verbose=true query parameter when you hit your prediction request. It throws the scores for all intents and then you can shortlist the top two or three.
Check sample here: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-concept-data-extraction

I tried to go this way and it does not work well at all. For example the intent is that I want to say that I have a dog. If I say I have a dog it gives me p=0.89, but when I say a bunch of random stuff and then say I have a dog in a separate sentence, the value is p=0.05. This makes it completely unreliable.
I tried to figure out how to split the input into sentences using punctuation, but LuisRecognizer.recognize wants context, not string input. I have no idea how to feed an array of things into it. any advice?