Botframework-composer: LUIS recognizer score condition with QnA Maker enabled doesn't work

[stephanbisser]

Describe the bug

When I use the "Luis & QnA Maker" template as it is and just insert my LUIS IDs and keys as well as my QnA Maker IDs and keys in the settings, it seems that QnA Maker is always winning.
The prebuilt BuySurface LUIS intent is getting added to my LUIS app but if I ask "how can i buy a surface pro?" in the emulator, I get "Sorry, I did not find an answer." which is coming from QnA Maker.
The strange thing I found out, is that the emulator trace shows first the LUIS trace and then the QnA trace:

and the LUIS trace results in a 0.9x confidence score:

within my Composer I have defined that the BuySurface score should be above 0.6:

If I remove the condition on the LUIS recognizer however, I get back the correct result coming from LUIS...

Version

What version of the Composer are you using? Paste the build SHA found on the about page (/about).

test

Browser

What browser are you using?

• [ ] Chrome
• [ ] Safari
• [ ] Firefox
• [x] Edge

OS

What operating system are you using?

• [ ] macOS
• [x] Windows
• [ ] Ubuntu

[hibrenda]

@stephanbisser Thanks for reporting the issue. I set up the LUIS&QNA template and defined the BuySurface score for 0.6 as well but I could not reproduce the issues. Could you share more detail for any specific changes you made ?

[stephanbisser]

I now created a brand new bot with the QnA and Luis template and just started the bot without modifying anything and get the same result:

How to solve that issue?

[hibrenda]

If you don't mind , could you please share your ComposerDialogs so we can further debug the issues.

[stephanbisser]

ComposerDialogs.zip

Here you go...

[hibrenda]

I tried out and find out the root cause. Wrong luis score set to "BuySurface" intent.
So every time you say "buy surface pro", you indeed get the top intent score with "BuySurface" but #help.score always < 0.8, so this trigger never been cached and then UnknownIntent will be triggered, that's the reason why QnA step will be called.

t cause.

[stephanbisser]

Sorry I provided you with the wrong ComposerDialogs. here are the correct ones:
ComposerDialogs.zip

[hibrenda]

could not reproduce the issue with your attached dialog resource.

[stephanbisser]

I guess I now found the error (which is a big problem) I have an English OS but I have set the locale to German. Therefore the OS translates a dot (".") to a comma (",") instead of keeping the dot. So this intent condition will never be fulfilled as it is always 0,5 instead of 0.5

That's a bit of a problem for everyone not using the English locale

[hibrenda]

@stephanbisser much appreciate your finding and reporting. I will also address your issue within the team as well.

[hibrenda]

close this as it has been root caused and no code change needed so far

[stephanbisser]

@hibrenda what is the solution to this issue now? Everyone in the non-English speaking market will get errors using the conditions within Composer?!

[hibrenda]

This is nothing about the composer, we will evaluate whether SDK runtime can solve it.

[cwhitten]

@stephanbisser this is a valid issue we're discussing solutions to right now. Thanks for your patience

[vishwacsena]

@stephanbisser I just tried to repro this on my English Win 10 machine and could not. Can you share crisp repro steps? What exactly do you mean by locale set to german? Do you mean Regional format set to 'English (Germany)' or Country or region set to 'Germany'? Also, I presume you were able to confirm that composer serializes down '0.9' as '0,9'? Thanks!

[stephanbisser]

I would like to but after I updated the Composer I get this errror whenever I start a bot:

Could not execute because the specified command or file was not found. Possible reasons for this include: * You misspelled a built-in dotnet command. * You intended to execute a .NET Core program, but dotnet-bin/Debug/netcoreapp2.1/BotProject.dll does not exist. * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.

Are you aware of that?

[stephanbisser]

I solved the issue with the execution now, but no I'm facing another error: Whenever I try to start a bot which has LUIS enabled I always get this in the emulator and I'm not able to communicate with the bot:

As soon as I remove the intent triggers from the bot I can chat with the bot. What's the issue here?

[stephanbisser]

Created a separate issue for that problem I'm facing currently: https://github.com/microsoft/BotFramework-Composer/issues/1965 if that's solved I can repro the steps for you @vishwacsena but in general I have had set everything to German (region, regional format and country - so the Windows display language was English but the rest was German and therefore "." got translated to "," as this is the common character for a comma in the German speaking area)

[vishwacsena]

@stephanbisser we've been unable to reproduce this issue on our end. @boydc2014 tried this on a brand new VM as well and was unable to repro.

Please reactivate with accurate repro steps to help us reproduce the issue on our end.

Note: This is probably related but we are planning to make the expression runtime be locale invariant. This has nothing to do with composer serializing content incorrectly though which is what I understand this issue to be. https://github.com/microsoft/botbuilder-dotnet/issues/3598

[dfvr1994]

Hey guys,

I encountered the same issue. I followed this tutorial https://docs.microsoft.com/en-us/composer/tutorial/tutorial-luis , and I wasn't able to get the "help" and "cancel" triggers to work unless I removed the #Help.Score >= 0.5 and #Cancel.Score >= 0.8 conditions.

Spanish (not working):

Changed the settings to decimal ( . ) and thousands ( , ) separations:

English (working):

Spanish (working):

So I guess is a matter of setting the decimal separator to point, right?

[Sbardo93]

Hi all,
I've found the trick.. simply change this trigger condition "#Intent.Score > 0.7" to "#Intent.Score*100 > 70".
I have set the locale to Italian and the trigger works!

[peterbozso]

@dfvr1994 Yes, that's it. I guess it's a couple of missing CultureInfo.InvariantCulture references here and there in the code of the generated chatbot.
I assume. I don't know the codebase enough (yet) to verify it myself. :)