Botframework-sdk: [LUIS] Help me understand how to integrate LUIS and QnA Maker in same app?

[fraangel]

Bot Info

• SDK Platform: Node.js
• Deployment Environment: Auzure Bot Service, local development with Emulator

Issue Description

Hi all! I cannot understand well the behaviour of integrating LUIS and QnA Maker. Right now I'm able to have both services running in my app. What I expected was that adding LUIS intent recognizer in my app it will be now able to answer kind of questions not completely in my QnA knowledgebase.

Just an example:
if I write to my bot: " who are you?" it will answer "Hi I'm your bot! "
instead if I write " tell me who you are" it recognize the LUIS intent related to the question but it will _not answer_ "Hi I'm your bot! " like I imagine.

So my question is : is it possible, combining LUIS and QnA Maker, to well answer user question even if these questions are not written in identical manners as in the QnA KB?

Hope my question is clear.
Thanks in advance

[JasonSowers]

Check out this blog and see if it helps you. It's in C#, but the concepts may help.

[fraangel]

@JasonSowers thanks for the answer! So I'll try to reword the question. Taking in example the blog post:
I ask a question to the bot --> it triggers an intent (luis) --> then is it possible to answer the user question using QnA KB ?
Get back to my example I have my QnA KB in which I have a pair : " who are you?" / "Hi I'm your bot! " . Then I have my Luis app that recognize this intent called "common" in which if I post questions :

• Who are you? --> common 0.97

• Tell me who you are --> common 0.92

So there's a way if I post to my bot : I ask the question "Tell me who you are" --> it triggers the intent common (luis) --> then I imagine it will answer me looking into the QnA KB --> "Hi I'm your bot! "

Is this scenario possible?

[JasonSowers]

If I am understanding you correctly, you can just call QnAMaker inside your "common" intent forwarding the query to QnA.

[nwhitmont]

Duplicate of https://stackoverflow.com/questions/48190871/bot-framework-integrating-luis-and-qna-maker-in-node-js?stw=2

[fraangel]

@nwhitmont it's not duplicate I wrote the same question on stackoverflow!
@JasonSowers can you please provide an example?

[nwhitmont]

@nwhitmont it's not duplicate I wrote the same question on stackoverflow!

Correct, you did, and that's why its marked as a duplicate. Posting the same question on GitHub and Stack Overflow _is the definition of a duplicate post_.

We monitor both forums concurrently. Please don't double post the same question on GitHub and Stack Overflow.

[fraangel]

Really sorry @nwhitmont , it will not happen again! Hope someone will answer

[JasonSowers]

var builder = require('botbuilder');
var restify = require('restify');
var cog = require('botbuilder-cognitiveservices');

var server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 3978, function() {
    console.log('%s listening to %s', server.name, server.url);
});

var connector = new builder.ChatConnector({
    appId: "APP ID",
    appPassword: "APP PASSWORD"
});
server.post('/api/messages', connector.listen());

var bot = new builder.UniversalBot(connector, function(session) {
    session.send('Sorry, I did not understand \'%s\'. Type \'help\' if you need assistance.', session.message.text);
});

var recognizer = new builder.LuisRecognizer("https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/{LUIS APP ID}?subscription-key={LUIS KEY}&verbose=true&timezoneOffset=0&q=");

bot.recognizer(recognizer);
var qnaRecognizer = new cog.QnAMakerRecognizer({
    knowledgeBaseId: 'QNA APP ID',
    subscriptionKey: 'QNA SUBSCRIPTION KEY'
}); 

bot.dialog('Common', function(session) {
    var query = session.message.text;
    cog.QnAMakerRecognizer.recognize(query, 'https://westus.api.cognitive.microsoft.com/qnamaker/v2.0/knowledgebases/{QNA APP ID}}/generateAnswer', '{QNA SUBSCRIPTION KEY}', 1, 'intentName', (error, results) => {
        session.send(results.answers[0].answer)    
    })    
}).triggerAction({
    matches: 'Common'
});

[GauravDhavale]

@JasonSowers : landed here while searching for building a hybrid model based on LUIS and QnAmaker.ai.

My understanding based on the above common dialog example is you need to have QnA related questions in LUIS so that LUIS can direct you to that dialog after identifying the intent.

I don't want to maintain the questions which can be answered using QnAmaker.ai inside the LUIS.
I referred this example to combine the recognizer and then execute the dialog as per recognizer match.

Can you please help me to understand, how can I differentiate the query and execute it to get the response?

Presently, I'm using the below flow to execute LUIS dialogs

bot.dialog('Policy', function(session) {
      //process request and get the response
        session.send(response) ;
}).triggerAction({
    matches: 'Policy'
});

So whenever the user sends something that matches to the policy intent, it should execute the above code and if the question belongs to a question from QnAmaker then I want to design a common function where I can redirect all QnAmaker related queries.

Shall I implement the QnAmaker.ai flow within LUIS None intent?

[Bhuvneshdadhich]

`require('dotenv-extended').load();

var builder = require('botbuilder');
var restify = require('restify');
var cog = require('botbuilder-cognitiveservices');

var server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 3978, function () {
console.log('%s listening to %s', server.name, server.url);
});

var connector = new builder.ChatConnector({
appId: process.env.MICROSOFT_APP_ID,
appPassword: process.env.MICROSOFT_APP_PASSWORD
});
server.post('/api/messages', connector.listen());

let bot = new builder.UniversalBot(connector);
var recognizer = new builder.LuisRecognizer("https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/?subscription-key=&verbose=true&timezoneOffset=0&q=")
var dialog = new builder.IntentDialog({ recognizers: [recognizer] });
var age;

bot.recognizer(recognizer);
var qnaRecognizer = new cog.QnAMakerRecognizer({

});

bot.dialog('/', dialog);

dialog.matches('greeting', [
function(session, args) {
session.send("Hello There! What can I do for you today?");
}
]);

dialog.matches('payment', [
function(session, args) {
session.send("sure enter your card details.");
}
]);

dialog.onDefault( [

function (session){

var query = 'How do I change my Bank Account details?';
cog.QnAMakerRecognizer.recognize(query, 'https://westus.api.cognitive.microsoft.com/qnamaker/v2.0/knowledgebases/{key1}}/generateAnswer', '{key}', 1, 'intentName', (error, results) => {
session.send(results.answers[0].answer)
})
}

]);

guys i am trying to combine luis and qna service in one bot, here i'm calling qna api in None intent . But I'm getting TypeError: Cannot read property 'answers' of undefined.
Can anyone help or suggest me any good method to do this?

`

[GauravDhavale]

@Bhuvneshdadhich: Well, you can approach in another way too.

var bot = new builder.UniversalBot(connector);

// QnA recognizer 
var qnaRecognizer = new cognitiveservices.QnAMakerRecognizer({
    knowledgeBaseId: QnA_knowledgeBaseId,
    subscriptionKey: QnA_subscriptionKey
});

// LUIS recognizer is used to read user message and convert it to intents using Microsoft LUIS
var luisRecognizer = new builder.LuisRecognizer(process.env.LUIS_MODEL_URL);

bot.recognizer(luisRecognizer);
bot.recognizer(qnaRecognizer);

// FAQ through QnAMaker
bot.dialog('qna', function (session, args) {
    var answerEntity = builder.EntityRecognizer.findEntity(args.intent.entities, 'answer');
    session.send(answerEntity.entity);
}).triggerAction({
    matches: 'qna'
});

[vishalkothari]

@GauravDhavale Where do you provide the QnA Maker Host URL in your approach?

[VanessaOrtiz30]

@GauravDhavale @Bhuvneshdadhich
How can I perhaps make this work in C#, any references?

[GauravDhavale]

@vishalkothari: I just provided knowledgeBaseId and subscriptionKey, nothing else.

var qnaRecognizer = new cognitiveservices.QnAMakerRecognizer({
    knowledgeBaseId: QnA_knowledgeBaseId,
    subscriptionKey: QnA_subscriptionKey
});

@VanessaOrtiz30: I have implemented only in Node. Found this article for C#.

[do-while-what]

@GauravDhavale
Did you find a solution to your problem (LUIS and QnA hybrid)?
I'm working on something similar.

[GauravDhavale]

@do-while-what: Yes, I posted the solution above.

[tsuwandy]

@VanessaOrtiz30, check out this doc/sample (SDK V4):
https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-tutorial-dispatch?view=azure-bot-service-4.0&tabs=csaddref%2Ccsbotconfig
https://github.com/Microsoft/botbuilder-dotnet/tree/master/samples-final/9.AspNetCore-Luis-Dispatch-Bot

[do-while-what]

@GauravDhavale
So you created a new intent called 'qna' in LUIS and are redirecting the user message to the qnarecognizer if the message is found to contain the qna entity by LUIS?

[GauravDhavale]

@do-while-what: No, I didn't create it. I'm just using it. qna is the default intent for qnamaker.

[praneethnooli]

@GauravDhavale @JasonSowers
Hello
I used @GauravDhavale approach https://github.com/Microsoft/BotBuilder/issues/4017#issuecomment-389710350 in integrating QnA maker with LUIS bot using sdk v3 Nodejs. Currently, I have many knowledge bases in my QnA maker and if a user asks a QnA question, depending on the corresponding knowledge base the question belongs to, I need to take an appropriate action (for example- some knowledge base responses need to be sent by email to the requested user rather than directly displaying the answer). Is there any way where I could get the knowledge base ID for the corresponding QnA question if it is asked? I don't want to create an intent for each of the knowledge base as there are many questions in each knowledge base and it is not feasible in creating an intent for each available knowledge base.

From the response given by QnA maker https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/how-to/metadata-generateanswer-usage#generateanswer-response, I could only get the id of the question but not the knowledge base id.

[EricDahlvang]

Would the 'source' property provide enough context?

If not, you could add the required info as metadata.

[praneethnooli]

Thanks, @EricDahlvang for your suggestion. I was able to use the metadata feature to solve my problem.

[fraangel]

Hi all,
since I wrote the question few things changed. First of all how to create and call the QnAMaker service.
The examples provided are no longer usable so, looking at what changed...now what's the best way to integrate LUIS and QnAMaker?
My idea is even the same: set INTENT and ENTITIES in LUIS and then query to the corresponding KB in QnAMaker.
Any suggestion to improve this flow? Should be still valid?
Thank

[neetu-das]

@Bhuvneshdadhich: Well, you can approach in another way too.

var bot = new builder.UniversalBot(connector);

// QnA recognizer 
var qnaRecognizer = new cognitiveservices.QnAMakerRecognizer({
    knowledgeBaseId: QnA_knowledgeBaseId,
    subscriptionKey: QnA_subscriptionKey
});

// LUIS recognizer is used to read user message and convert it to intents using Microsoft LUIS
var luisRecognizer = new builder.LuisRecognizer(process.env.LUIS_MODEL_URL);

bot.recognizer(luisRecognizer);
bot.recognizer(qnaRecognizer);

// FAQ through QnAMaker
bot.dialog('qna', function (session, args) {
    var answerEntity = builder.EntityRecognizer.findEntity(args.intent.entities, 'answer');
    session.send(answerEntity.entity);
}).triggerAction({
    matches: 'qna'
});

this works

[tsuwandy]

Also, take a look at this doc/sample:
https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-tutorial-dispatch?view=azure-bot-service-4.0&tabs=csharp