Botframework-solutions: Replaced deprecated LuisPredictionOptions with LuisRecognizerOptionsV3

[rvallireply]

I used the following command to generate the GeneralLuis.cs class from a LUIS app JSON set to the italian it-it locale:

bf luis:generate:cs -f -i "C:\users\johndoe\source\repos\myskill\Deployment\Resources\LU\it-it\VMyskillit-it_General_v0.1.json" -o "C:\users\johndoe\source\repos\Myskill\Services\" --className="GeneralLuis"

ISSUE: when I test the bot with italian language and activities like "apri un ticket" ("open a ticket") an exception is thrown at the following line in Convert function

var app = JsonConvert.DeserializeObject<GeneralLuis>(JsonConvert.SerializeObject(result, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }));

Exception:
Newtonsoft.Json.JsonReaderException: 'Could not convert string to double: un. Path 'entities.number[0]', line 1, position 183.'

If I change "un" to "one" at runtime the exception is not thrown.
It seems LUIS correctly translates "un ticket" (italian article for "a ticket") to "one" and passes it as a number but the resulting serialized JSON keeps the string "un" instead of the number, hence the type mismatch

{"text\":"apri un ticket","intents":{"Stop":{"score":0.045692727}},"entities":{"$instance":{"number":[{"startIndex":5,"endIndex":7,"text":"un","type":"builtin.number"}]},"number":["un"]}}

This doesn't happen only for italian indefinite articles "un, uno, una" ("a", "an").
The same of course happens if I tell the bot to "apri due ticket" ("open two tickets") and so on...

[munozemilio]

Thanks for reporting this @rvallireply, I will tag it to be addressed in the current release cycle.

[vishwacsena]

@rvallireply R8 is expected to land mid March 2020.

[chrimc62]

This looks to be an issue for https://github.com/Microsoft/Recognizers-Text and not the SDK. The italian number recognizer is not generating a resolution. @vishwacsena can you do whatever magic we need to move this to that repo and link appropriately? We should also make sure the LUIS team is aware--it might be fixed and they have not picked up the latest library.

[vishwacsena]

@chrimc62, I sent mail to text entity recognizer and LUIS PMs. I unfortunately do not have the right privileges to transfer this over to the recognizers repo.

[tellarin]

Hi folks,
I can't reproduce this in Recognizers-Text. The response I get includes the proper resolution.

{
  "Text": "un",
  "Start": 5,
  "End": 6,
  "TypeName": "number",
  "Resolution": {
    "value": "1"
  }
}

@vishwacsena, in which version and platform did this happen?
For similar feedback that may be related to Recognizers-Text, feel free to open an issue there and link to the original one; or let me know what kind of permission you need to transfer and we can try to set that up.
Thanks.

[vishwacsena]

@rvallireply - can you clarify what LUIS API version are you using?

[rvallireply]

@vishwacsena currently v3

[chrimc62]

I am unable to repro this in V3, but it does happen in V2. Does your code look something like this:

            var app = new LuisApplication(<appID>, <appKey>, "https://westus.api.cognitive.microsoft.com");
            var options = new LuisRecognizerOptionsV3(app);
            var recognizer = new LuisRecognizer(options);

In particular LuisRecognizerOptionsV3?

[rvallireply]

No, this is the code I found using in the VA template BotServices.cs that I didn't touch:

var luisOptions = new LuisPredictionOptions()
                {
                    TelemetryClient = telemetryClient,
                    LogPersonalInformation = true,
                };

                var dispatchApp = new LuisApplication(config.DispatchModel.AppId, config.DispatchModel.SubscriptionKey, config.DispatchModel.GetEndpoint());

                set.DispatchService = new LuisRecognizer(dispatchApp, luisOptions);

                if (config.LanguageModels != null)
                {
                    foreach (var model in config.LanguageModels)
                    {
                        var luisApp = new LuisApplication(model.AppId, model.SubscriptionKey, model.GetEndpoint());
                        set.LuisServices.Add(model.Id, new LuisRecognizer(luisApp, luisOptions));
                    }
                }

[chrimc62]

That is using V2 of the LUIS API and why you are having this issue. Change LuisPredictionOptions to LuisRecognizerOptionsV3 and your issue should be fixed.

[cleemullins]

@darrenj Per the thread, the default BotServices.cs in the VA needs to be updated to replace the LuisPredictionOptions to use the v3 Endpoint (see @chrimc62 reply above). Xferring issue to repo for tracking.

[darrenj]

Thanks @cleemullins - yes we addressed this a few weeks back and is in our next branch to go out in next release. I'll add a known issue in the docs too

[darrenj]

The update away from the deprecated feature was addressed in #3061 however this is still V2. @pavolum could you look into updating to the v3 endpoint addressing this issue and moving us to the latest.

@cleemullins This should probably be covered in BF docs somewhere (i.e. the differences between V2 and V3 and what known issues there are for customers wiring up LUIS themselves? Deprecation warning too perhaps?

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days