Botframework-webchat: Custom Speech Service: CRIS Integration

[vsusloparov]

Hello to everyone,

Today we have project that uses WebChat in client side, LUIS and other MS services from server side. We need to add the custom speech service (CRIS) to our project.

Is it possible to use build in functionality of "BotFramework-WebChat" and switching usage of Bing service or to CRIS service? Is it supported?
In case it's not, is it will be supported in a future?

Reference to code:
import * as CognitiveSpeech from 'microsoft-speech-browser-sdk/Speech.Browser.Sdk'

Information about the service:
Custom Speech Service
CRIS.AI

[compulim]

@vsusloparov. Yes, we support writing a custom speech recognizer to work with Web Chat.

Look at SpeechModule.ts. You need to implement a ISpeechRecognizer for CRIS.

Since HTML LS also have a speech recognition API, I would do the following:

• Make an adapter between CRIS API and HTML SpeechRecognition API
  
  
  
  • On the upside, you can open source your package
  
  
• In Web Chat, our BrowserSpeechRecognizer actually consume HTML SpeechRecognition API, with a few line change (or refactoring), you can easily adapt it to your adapter

Please let me know if you need help on the topic. We are happy to help.

[vsusloparov]

@compulim. Thank you for response.

We will check the integration solution that you described above.

[ryanvolum]

@vsusloparov I need the same functionality - did you end up developing something? I'll probably get started on something and contribute it back.

[Tamoormughal]

I am working on Custom speech in bot framework.I add Bing simple speech it is working on my website: here is link :https://envolvebot.azurewebsites.net/

but now i want to know how can we add custom speech (LUIS custom speech) in our webchat bot? I create custom speech using : https://westus.cris.ai/Home/CustomSpeech Now i want to add it and want it to work like bing speech .

aNY BODY Know how to do this ?

Thank in advance .

[techie921]

Hi @Tamoormughal,

I have implemented Bing speech in one of my project but it is not working. Could you please provide working code base to me.

Thanks in Advance

[Tamoormughal]

@techie921 send me code i will update your code

[Tamoormughal]

@techie921
var speechOptions;
speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: 'key' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: 'key',
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
})
};

[techie921]

@Tamoormughal
The Same code we used for invoking the Speech Service from webchat but when i click the Mic Button in webchat its not being invoked. No Response or action taking place. help in this regard would be greatly appreciated.

[Tamoormughal]

@techie921
publish URL must be https not http
if this is not working then send me that part of code

[techie921]

We have tried with https, still it is not working. When we keep BrowserSpeechRecognizer and sent voice command then it is working. But when we keep below code with same configurations, it is not working and mic button is not enabled and getting websocket connection is failed in WebSocketMessageAdapter.ts file. Any help on this will be appreciated.

const speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: 'key' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: 'key',
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
})
};

[Tamoormughal]

@techie921
send me url where you publish it

[techie921]

We are running it from local tomcat by deploying index.html (webchat)

[techie921]

@Tamoormughal
We are getting below error while clicking on mic button. Any help on this will be appreciated.
WebSocket connection to 'wss://speech.platform.bing.com/speech/recognition/interactive/cognitiveservices/v1?format=simple&language=en-US&Ocp-Apim-Subscription-Key=6f9971474e0041b7aa40d122dc0fb145&X-ConnectionId=AF89B5FFD67145929DD038EDC475FC65' failed: Error during WebSocket handshake: Unexpected response code: 403

Thanks in advance.

[Tamoormughal]

@techie921 hope your key is valid

[techie921]

We are using Speech service API key here. We don't have Bing Speech service API Key. We are unable to create it.

[Tamoormughal]

@techie921 kindly use the azure services .and WebSocket turn into true from index.html

[techie921]

@Tamoormughal
Could you please drop your contact no to [email protected] as we are unable to resolve the issue.

Thanks in Advance.