Botframework-webchat: Bug report for speech option to be enabled on safari browser for Mac OS and iOS devices

[Bradrajkumar]

Very first thing, this is for webchat V4.

here is the link i followed for enbaling the speech option in safari browser:-
https://github.com/microsoft/BotFramework-WebChat/blob/master/SPEECH.md#special-considerations-for-safari-on-mac-os-and-ios

There is one another link which i tested on iOS mobile device safari and Mac OS safari browser (a sample bot provided by your team):- https://microsoft.github.io/BotFramework-WebChat/06.c.cognitive-services-speech-services-js/
observations:-

1. on iOS phone safari browser mic is visible.
2. on Mac OS safari browser mic is not visible .

In the article provided by team, it is written that "You can present a splash screen with a tap-to-continue button, which will ready the engine by sending an empty utterance to unblock Safari."
i want code snippet to handle it which i am not getting how to achieve it for safari browser only?(not using react)
i have also tried to see related options on stackoverflow but didn't find anything relavant for webchat V4.

CONTEXT 1 : - iOS phone safari browser

---

These are the observations in iOS phone safari browser with your reference bot, please look into it :-

1. if i say "hi", bot is responding with text but no speech.
2. if i say "help" bot is responding with series of responses and again mic will become available to talk and due to this i again clicked on mic and said "hi".
3. when the 2nd time i said "hi", bot is responding with text and still no speech.
4. after response of 2nd time "hi" the mic got disabled permanently.

i am using iphone7 plus and build version is 12.1.4 (not sure about others iOS devices)
getting unsure about the functionality/behaviour of mic option in iOS

CONTEXT 2 : - Chrome on windows desktop application

---

With your reference bot, if i opened it on chrome browser here are the observastions: -

1. if i say "hi", bot is responding with different text and different speech ----- here question is why different text and different speech.
2. if i say "help" bot is responding with series of responses and again mic will become available to talk and due to this i again clicked on mic and said "hi".
3. when the 2nd time i said "hi", bot is responding with different text and different speech.
4. but everytime bot is responding and mic is available.

---

Approach i have tried to embed the webchat V4 and i was successful to add the mic option and it is also visible on safari browser for iOS Device and Mac OS. the situation here is the behaviour is not as expected like in chrome windows desktop application. after one response mic is getting disabled on safari browser in both iOS and Mac.

here is my final code which i have written, working fine on chrome :-

(async function(){
const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate', {method: "POST",headers: {Authorization:"Bearer + DirectlineConnection}});
const {token} = await res.json();
const webSpeechPonyfillFactory = await window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
region: 'specific region',
subscriptionKey: "subscription key"
});
const store = window.WebChat.createStore();
window.WebChat.renderWebChat({
directLine: window.WebChat.createDirectLine({token}),
webSpeechPonyfillFactory,
store,
userID: "user ID",
username: "username",
locale: "en-US"
},document.getElementById('webchat'));

/inside this area i have also added the customizations like appending header, custom buttons inside webchat message panel with custom click event to send message to bot directly, etc./

document.querySelector('#webchat > *').focus();
})().catch(err =>console.error(err));

this is the code which i have finalized to use in my webhchat js file downloaded from CDN link.

Finally, in Chrome it is working and i want the expected behaviour to be on both iOS as well as Mac OS.
please review the bug report and provide me the descriptive information about the possiblities of handling speech in safari browser for Mac and iOS.
i am eagerly waiting for reply.

(Edited by @compulim for code formatting)

[carlosscastro]

@compulim and / or @tdurnford could you please take a look? Thanks!

[tdurnford]

I'm going to try and reproduce this on my mac later today.

[tdurnford]

@Bradrajkumar is correct. Speech in Web Chat v4.5.2 does not work in Safari; however, it seems to be fixed in the latest build. This should be resolved in our next release towards the end of October. If you need an immediate solution, please take a look at our documentation on using the Latest Bits. Thanks for reporting this issue!

[Bradrajkumar]

thank you @tdurnford for considering the issue , eagerly waiting for reply from your end.

[corinagum]

This bug has been fixed and will be available in our next release in October. If you need these changes immediately, please see our documentation on pointing to our MyGet feed for latest bits.

[Bradrajkumar]

@corinagum please do let me know the fixes done and releases with feature in speech service. please do let me know where i can see the changes done.

[compulim]

Hi @Bradrajkumar could you verify if this is fixed in Web Chat 4.6.0?

https://github.com/microsoft/BotFramework-WebChat/releases/tag/v4.6.0

[Bradrajkumar]

@compulim thank you for your quick reply.
i just want an another information that does these changes are already added to webchat latest version?
"https://cdn.botframework.com/botframework-webchat/latest/webchat-es5.js"