Botframework-sdk: Webchat bot messages are late - arrive only after next user message

[alexsorokoletov]

Hi!
Having this simple bot shows a problem with webchat (code below)
Bot has several waterfall-like prompts and then sends a message after a data processing is done.
Also bot has the "reset" command for easy debug.

Using this bot with a webchat shows that the bot response (message) comes only after user sends a new message to bot.
You can see the issue after I send a Hi and Hello? messages to bot.
After Hi I should receive a first name question but I didn't get anything from bot. I replied Hello? and in then end you can see bot accepted Hello? as an answer to first name question even though visually question came later.

This is the test transcript of the chat:

>reset
You · 1 min ago

>Sorry. The service was upgraded and we need to start over.
AlexDevBot · 1 min ago

>Hi
You · Now

>Hello?
You · Now

>Hi! What is your first name?
AlexDevBot · Now

>Alex
You · Now

>Hi! What is your last name?
AlexDevBot · Now

>Sorokoletov
You · Now

>Hi! What is your middle name?
AlexDevBot · Now

>AA
You · Now

>Please wait, processing your answer (5s)
AlexDevBot · Now

//after 10 seconds
>5 seconds passed
You · Now

>Hi! What is your first name?
AlexDevBot · Now

>Processed your answer 'Hello? Sorokoletov Alex', thanks.
AlexDevBot · Now

This is the repro bot code, running latest master.

/*-----------------------------------------------------------------------------
A simple "Hello World" bot for the Microsoft Bot Framework. 
-----------------------------------------------------------------------------*/

var restify = require('restify')
var builder = require('../../core/')

// Setup Restify Server
var server = restify.createServer()
server.listen(process.env.port || process.env.PORT || 3978, function () {
  console.log('%s listening to %s', server.name, server.url)
})

// Create chat bot
var connector = new builder.ChatConnector({
  appId: process.env.MICROSOFT_APP_ID,
  appPassword: process.env.MICROSOFT_APP_PASSWORD
})
var bot = new builder.UniversalBot(connector)
server.post('/api/messages', connector.listen())

// =========================================================
// Bots Dialogs
// =========================================================
bot.use(builder.Middleware.dialogVersion({ version: 1.0, resetCommand: /^reset/i }));


bot.dialog('/', [
  function (session) {
    builder.Prompts.text(session, 'Hi! What is your first name?')
  },
  function (session, results) {
    var firstName = session.userData.firstName = results.response
    builder.Prompts.text(session, 'Hi! What is your last name?')
  },
  function (session, results) {
    var lastName = session.userData.lastName = results.response
    builder.Prompts.text(session, 'Hi! What is your middle name?')
  },
  function (session, results) {
    session.send('Please wait, processing your answer (5s)')
    var firstName = session.userData.firstName 
    var lastName = session.userData.lastName
    var middleName = session.userData.middleName = results.response
    var callbackAddress = session.message.address
    setTimeout(function () {
      var msg = new builder.Message()
        .address(callbackAddress)
        .text("Processed your answer '%s %s %s', thanks.", firstName, middleName, lastName)
      bot.send(msg)
      session.endDialog()
    }, 5000)
  }
])

Gif showing the problem is attached

[changhc]

Hi @alexsorokoletov , where did you deploy your bot? On Azure Web App?

[OmShrivastava]

Hi I am also facing same issue in my chat bot app I have hosted in Azure web app. and to chat I am using Skype and web chat

[alexsorokoletov]

I have this both in Azure and with ngrok

[OmShrivastava]

Hi alex
In my application this issue is resolved after updating bot.builder to 3.1.0, now everything working fine

[alexsorokoletov]

Doesn't help. As I said I'm using the latest from GitHub

[danmarshall]

Hi @alexsorokoletov, setTimeout is not a supported feature from within a dialog in the Node SDK. Closures do not truly behave as you'd expect in a typical Node app. In your scenario, you may be trying to message the user proactivley, see this issue which contains insight on how to do that: https://github.com/Microsoft/BotBuilder/issues/431

[alexsorokoletov]

@danmarshall, thanks for noting this. I will definitely take a look. As you can see from the gif and conversation log, problem appears before bot even gets to the last step.

After I write him Hi I should get some response from bot. I do not receive it until I send another message ("Hello?").

[alexsorokoletov]

Also, what specifically I should look at #431? There is no information about 'setTimeout'

[danmarshall]

Correct, there is no information about setTimeout. I am guessing that you want to send something to your user at a later time? Or are you trying to simulate a long running process?

[alexsorokoletov]

Dan, this issue is not about set timeout. It's about incorrectly working webchat channel. The webchat doesn't send message from bot to user until user sends a next message.

[danmarshall]

Do you see the same behavior if you remove your setTimeout?

[changhc]

This behavior is indeed not related to setTimeout, because it happens even if the bot is just a simple copycat bot.

I've tried upgrading my azure app service from free to standard tier, but this doesn't really help. I guess that there's some message timeout settings in the SDK, and if it takes too long to send messages back or to generate reply messages, they will be kept until the next trigger.

If possible, you can talk to this bot via Skype as a temporary substitute. It works normally.
Or maybe you can try to deploy it on a different server. Perhaps it helps :P

[alexsorokoletov]

We have tried this on different azure servers, 4 total, still happens no matter what. Happens only on webchat channel, same bot and deployment works fine over Skype, Facebook and Telegram. I think we had similar problem before with Telegram (might be wrong) and that was fixed in the SDK update.

[alexsorokoletov]

Hi guys, do we have any update on this? We can't use webchannel because of this bug. I've put time and created a sample bot that reproduces the problem, recorded conversation and gif showing the problem. What else do we need from me to move on?

[danmarshall]

Thanks @alexsorokoletov for filing the issue, I have verified that there is a bug here. It does not have anything to do with setTimeout. Thanks we will be looking into it.

[tefrati]

Hi guys,

Any update? Same issue here with my WhoAreYou bot.
BTW, I'm using AWS and not trying to proactively send something to the user, so, definitely, an issue with the web client.

Thanks for your help!
Zach

[danmarshall]

Thanks all for your patience. We are actively working on an update for the webchat control, and more importantly, a new protocol for DirectLine which should ensure message order. We do not have an ETA but there is a lot of demand for this so keep an eye on our blog for updates, thanks!

[alexsorokoletov]

Hi guys! Any updates on that? There is a lot of work for you to do, I know, on the other hand this stops a lot of devs from publishing a bot on their sites. We're waiting for a fix for like a month (I created an issue after realized my request about this fix was lost). Any ideas how we could fix this? Maybe we could fix it and make a pull request?

[tefrati]

Any updates?
Our users are not as patient as we are. Here are some of the texts I'm getting from users trying the web channel:

"this sucks"

"lag"

and my favorite,

"What technology powers this crappy bot"

Any ETA on fixing the bug?

[danmarshall]

Hi folks, thanks for your patience and enthusiasm. Just want you to know that we're listening! But I have nothing to add to my previous message. We're actively working on this and want to present a great solution for the community, so please stay tuned. -Dan

[naeemshaikh27]

Hello guys, any update on this. This issue occurs a lot on webchat. desperately waiting for a solution.

[dandriscoll]

Hi @naeemshaikh27, we have a bugfix going out soon that will fix this.

[ubreddy]

Its almost a month since being reported @dandriscoll. this is a critical issue rendering the channel redundant till it is fixed. is there a date by which we can expect this fix?

[dandriscoll]

The fix went out a few days ago. You're still seeing problems?

[ubreddy]

Confirming this is fixed. Thanks a lot....

[changhc]

May I ask how you fixed it?
Thanks.

[dandriscoll]

This was a bug in the JavaScript within the chat control.

[alexsorokoletov]

Confirming this is closed. Thank you guys!

[arieschwartzman]

Still seeing this problem

[naeemshaikh27]

yes still see the problem. It is a little different than before now. Previously the messages only arrived one at a time only one step behind after each user input. now the messages arrive together when the lag, but only after a user input.

Example:
user:- Hi
bot:- hi, my name is msbot, i can help you with a bing search, please tell me what you want to search?
user:- what is LUIS-----(1)

====No response====
user:- are u there?---(2)
bot:- LUIS is----bing search result---(1)
bot: yes i m here to help you, how can i help you---(2)

So is this still fixed properly?

[dandriscoll]

Hi @arieschwartzman and @naeemshaikh27, this is being tracked under a different issue. #965

[pandubalraj]

Hi @dandriscoll ,

I am facing the same issue.
While entering the second chat message then only I am getting the first chat response.
Still having the same as @alexsorokoletov mentioned in first GIF image.

I have deployed in Azure and using the NodeJS SDK botbuilder version 3.2.3.

Please let me know, is this fixed ?
Thanks