Botframework-webchat: Cognitive Speech not working in West Europe

Created on 7 Jan 2020  路  21Comments  路  Source: microsoft/BotFramework-WebChat

Screenshots

image


Version

I use 4.7.1, but it doesn't work on 4.6.0 too.

"<meta name="botframework-directlinespeech:version" content="4.7.1">
<meta name="botframework-webchat:bundle:variant" content="full-es5">
<meta name="botframework-webchat:bundle:version" content="4.7.1">
<meta name="botframework-webchat:core:version" content="4.7.1">
<meta name="botframework-webchat:ui:version" content="4.7.1">"

Describe the bug

When I click on the microphone this error occurs.

On 19.12.2019 everything worked fine and when I tried on 20.12.2019 WITHOUT changing anything at the code it wouldn't work.

This is how I implement the Cognitive Speech Service:

<script>
webSpeechPonyfillFactory = window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
            //  credentials: {
                        region: 'westeurope',
                        subscriptionKey: 'XXXXX'
            //      }
            })
            window.WebChat.renderWebChat({
                directLine: window.WebChat.createDirectLine({
                //  token: token
                    secret: 'XXXXXXXX',

                }),
                webSpeechPonyfillFactory: webSpeechPonyfillFactory,
                // disable timestamp
                groupTimestamp: false,

                //set language
                locale: 'en-US',

            }, 
                document.getElementById('webchat'));


      </script>


Expected behavior

It should recognize my voice and translate it to text.

Additional context

My bot runs on C#, but I tried on a JS-Bot too, same Error. WebSockets are enabled in the BotConfig.

[Bug]

Bot Services Bug Cognitive Services customer-replied-to customer-reported
Source
picture wehnicde

Most helpful comment

Although you may not using LUIS in your bot, Direct Line channel do provides a LUIS ID and requires Web Chat to pass it to Cognitive Services. The ID is called "LUIS reference grammar ID".

Even in your case, you are not using LUIS, Direct Line channel still send us the LUIS ID, so we still pass it to Cognitive Services. And I believe Cognitive Services has some issues handling LUIS ID in West Europe region.

We have contacted Cognitive Services team and they are investigating the issue.

picture compulim on 16 Jan 2020
👍3

All 21 comments

Hi @wehnicde, I have been unable to repro your issue. I have a couple clarifying questions for you:

  • Which browser are you running Web Chat in? It looks like you are using Chrome but you are utilizing the webchat-es5.js package (vs. the regular webchat.js package). Can you try the regular package?
  • Is this the React version of Web Chat? Your code suggests it isn't, but the rendered web chat component in your image looks a lot like the "minimizable" version which is built in React.
  • There were some issues with the 4.7.1 build that have come to light recently and will be corrected in 4.7.2. It has been suggested you try build 4.5.3. Can you give this a go and see if it helps?
  • In createDirectLine(), can you try passing webSocket: true, as well?
stevkan picture stevkan on 8 Jan 2020

Which browser are you running Web Chat in? It looks like you are using Chrome but you are utilizing the webchat-es5.js package (vs. the regular webchat.js package). Can you try the regular package?

I am running in Chrome but I need IE support too, thats why I use es5. (Does it have any disadvantages?)
I tried the normal package too, but it doesn't work either.

Is this the React version of Web Chat? Your code suggests it isn't, but the rendered web chat component in your image looks a lot like the "minimizable" version which is built in React.

It isn't. The "taskbar" on top is simple html by me (using style.display = none), it is the standart webchat.

There were some issues with the 4.7.1 build that have come to light recently and will be corrected in 4.7.2. It has been suggested you try build 4.5.3. Can you give this a go and see if it helps?

Didn't help :( When I change to 4.5.3 the webchat won't render with the error: 

TypeError: e is not a function
    at t.<anonymous> (webchat.js:1)
    at t.createWebSpeechPonyfill (webchat.js:1)
    at t.value (webchat.js:1)
    at Io (webchat.js:1)
    at To (webchat.js:1)
    at Mo (webchat.js:1)
    at Ga (webchat.js:1)
    at Ka (webchat.js:1)
    at Is (webchat.js:1)
    at As (webchat.js:1)

If I take the speech stuff out, it works. With 4.6 it is rendering, did 4.6 change anything about the implementation of webSpeech?

This is my complete HTML-Code (clean version).

<!DOCTYPE html>
<html>
  <head>
    <title>ChatBot CleanHTML</title>
  </head>
  <body>            
    <div id="webchat" role="main"></div>
    <script 
      crossorigin="anonymous"
      src="https://cdn.botframework.com/botframework-webchat/4.5.3/webchat.js"
    ></script>

    <script>            
      window.WebChat.renderWebChat({
        directLine: window.WebChat.createDirectLine({
          webSocket: true,
          secret: 'XXXXXXXX',
        }),
        webSpeechPonyfillFactory: window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
          region: 'westeurope',
          subscriptionKey: 'XXXXXXXXX'
        }),
        locale: 'en-US',
      }, 
    document.getElementById('webchat'));
    </script>
  </body>
</html>

In createDirectLine(), can you try passing webSocket: true, as well?

Didn't change anything.

I have been unable to repro your issue.

I will try to generate a fresh bot and try to repro it by myself again. My problem is, that I already have 2 seperate bots (C# and JS) and it isn't working on both, so I really don't know what the problem could be.

Edit: Just generated a clean bot and put the secret in my html. Didn't work. So the bot-code shouldn't be the problem.

wehnicde picture wehnicde on 8 Jan 2020

@wehnicde, I'm a little confused on whether you are trying to get this to work in Chrome, IE, or both. If you are having issues with Chrome, I'm certain we can get it working. Your implementation is slightly different but really should work. And, if not, we can play with the code.

As for IE 11, as stated here, Internet Explorer does not meet the minimum requirements for supporting speech recognition and speech synthesis, at least as far as Cognitive Services Speech is concerned.

stevkan picture stevkan on 11 Jan 2020

Sorry for the confusing. I want to have the webchat working on all browsers, but I know that speech isn't possible with IE11. Writing with the bot should work on every browser, speech only with Chrome/Firefox. Thats why I need to implement the es5 file (Am I right about that?)

What do you mean by "Your implementation is slightly different but really should work"? How is my implementation different? And how is the "normal" way? Maybe it will work with that.

wehnicde picture wehnicde on 13 Jan 2020

I am getting the same problem in IE, Chrome, Edge Chrome.
webchat.js:2 WebSocket is already in CLOSING or CLOSED state.
I can provide a URL to a test page that shows the issue if you want. Any chance to DM that?
I'm still using the token generation to make it work. It did work but no longer does so some change was pushed on your end that broke it (we pushed no changes)

I should note that the bot still works in Chrome for typing, when you press the microphone for speech it does ask to allow, then the errors start pouring in and no actual listening seems to happen.

markbschramm picture markbschramm on 15 Jan 2020

Adding my voice: getting same error in Chrome, a couple days ago Speech was working, now without changing anything in the code I'm getting webchat.js:2 WebSocket is already in CLOSING or CLOSED state.
Using westeurope region too.

I noticed this issue was just closed (10 hours ago): anything to do with it maybe? (regression?)
https://github.com/microsoft/BotFramework-WebChat/issues/2459

rvallireply picture rvallireply on 15 Jan 2020

@wehnicde, yes, the es5 version will work in all web browsers the same. Outside of the file package size if that is a concern, you should be just fine.

Never mind my comment about implementation. You're is just fine and acceptable. I thought I saw you doing something slightly different, but I was mistaken.

I did go ahead and create a new CS Speech service a second time (and a new bot for a clean slate tho I also confirmed with an established bot) and was able to repro. I will have an update when we know more.

stevkan picture stevkan on 15 Jan 2020
👍2

Thanks @stevkan for the repro. Adding @compulim to the thread.

corinagum picture corinagum on 15 Jan 2020

Reassigning @stevkan for azure reporting only

CoHealer picture CoHealer on 15 Jan 2020
👍1

This bug is very similar to https://github.com/Azure-Samples/cognitive-services-speech-sdk/issues/127.

West Europe services is returning "Specified grammar type is not supported!" when we pass LUIS reference grammar ID.

image

image

When we take away LUIS reference grammar ID (in hacked code), it works with West Europe.

So I am assuming Cognitive Services in West Europe is having issue on handling LUIS reference grammar ID.

@stevkan could you file a ticket to them, mentioning LUIS reference grammar ID (a.k.a. IntentRecognizer) is not working West Europe? I have pinged @mahilleb-msft offline, but he is in Munich time zone, no response yet.

compulim picture compulim on 16 Jan 2020

@compulim I am not using LUIS for my bot. Or does Cognitve Speech use LUIS by itself?

wehnicde picture wehnicde on 16 Jan 2020
👍1

Although you may not using LUIS in your bot, Direct Line channel do provides a LUIS ID and requires Web Chat to pass it to Cognitive Services. The ID is called "LUIS reference grammar ID".

Even in your case, you are not using LUIS, Direct Line channel still send us the LUIS ID, so we still pass it to Cognitive Services. And I believe Cognitive Services has some issues handling LUIS ID in West Europe region.

We have contacted Cognitive Services team and they are investigating the issue.

compulim picture compulim on 16 Jan 2020
👍3

Oh okay. Thanks for the clarification. And thanks for the investigation.

wehnicde picture wehnicde on 16 Jan 2020

@compulim, created a ticket earlier today in Cognitive Services Speech Uservoice account, here.

stevkan picture stevkan on 17 Jan 2020
👍1

@stevkan The error seems different today. Not sure if you changed anything, I did not: (It's similar)
3SCRIPT12002: SCRIPT12002: WebSocket Error: Network Error 12002, The operation timed out

markbschramm picture markbschramm on 17 Jan 2020

Web Chat only own the client-side. I believe the service team is working on a fix.

compulim picture compulim on 17 Jan 2020
👍1

I'm receiving a similar issue but my Speech is set to the UK South region. Would that also be affected by this?

phenaxkian picture phenaxkian on 17 Jan 2020

Hi, I am getting this same WebSocket issue in locale:en-US and speech region: eastus

Is there any fix for this? We are not able to use speech service because of this.

alokraj68 picture alokraj68 on 17 Jan 2020
👍1

Any news about the "WebSocket is already in CLOSING or CLOSED state." issue? I tried with northeurope and it seems to fail too.
By the way, I am now confused about the right javascript code to use when connecting to cognitive speech. I saw a warning "We are deprecating authorizationToken, region, and subscriptionKey. Please use credentials instead." Could someone provide an up-to-date sample about how to use webchat with cognitive speech service today, please?

rvallireply picture rvallireply on 20 Jan 2020

Closing this issue as it is working now. Thanks for the investigation.

wehnicde picture wehnicde on 22 Jan 2020
🎉1 👍1

Thanks @wehnicde for reporting the problem.

compulim picture compulim on 22 Jan 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

corinagum picture corinagum  路  3Comments
vikramdadwal picture vikramdadwal  路  3Comments
filipjakov picture filipjakov  路  4Comments
adriantan08 picture adriantan08  路  3Comments
AndreMantas picture AndreMantas  路  4Comments