Botframework-webchat: Only a single Adaptive Card can render before additional user input

[jherres]

In this specific customer scenario, they wish to send 2 different adaptive cards (in sequence not a carousel) as a response to a user. I have found that when the second card is sent, the first will be removed from chat.

In addition to this, no additional adaptive cards will render until the user sends some other command. For example, I click on a submit button in the adaptive card which will trigger the bot to do the exact same skill, no adaptive cards will render. Once the user sends text to the bot, then we get back into the loop of the second adaptive card removing the first.

Here is a quick video of the issue https://youtu.be/pXocEbtEDxg

Edit: This issue does not exist in V3 and we had to roll that customer back to V3 for now.

[corinagum]

~Hi @jherres, let me get back to you after I reproduce this issue.~
turns out @compulim is already working on this :)

[compulim]

I am trying to repro but no luck, I am sure there are something missing. In my bot code:

await context.sendActivity({
  type: 'message',
  text: '# This is some text\n1. this is this first item\n2. the second\n3. last one'
});

await context.sendActivity({
  type: 'message',
  attachmentLayout: 'carousel',
  attachments: [{
    contentType: 'application/vnd.microsoft.card.adaptive',
    content: { ... }
  }]
});

await context.sendActivity({
  type: 'message',
  text: 'This is some more text for you to see'
});

await context.sendActivity({
  type: 'message',
  attachmentLayout: 'carousel',
  attachments: [{
    contentType: 'application/vnd.microsoft.card.adaptive',
    content: { ... }
  }]
});

I am seeing 4 activities coming from the bot, both Adaptive Cards are showing at the same time and they never get lost.

Could you write up a minimal repro to see if anything weird on your side?

This file and this line is where we might remove/overwrite some of the activities. It will be great if you can spot any wrongdoers in this code.

[jherres]

I am working to make V4 compatible with our no-code bot building platform (AtBot). I am sure we are doing some things outside of the norm. Let me loop in someone who is more familiar with the actual bot code (I can only see what comes over via developer tools or fiddler).

[jherres]

They will be able to have a look later this evening. But for now, I can provide the fiddler trace of the websocket traffic for those cards. I have attached the full fiddler archive of the interaction, but the two cards are as follows:

DroppedCards.zip

First Card
{ "activities": [ { "type": "message", "id": "FZ0T3tDxqgr7jAax8IuOKv-j|0000004", "timestamp": "2019-05-03T19:53:29.1256488Z", "channelId": "directline", "from": { "id": "askflowbizzybot", "name": "Ask SPTechCon" }, "conversation": { "id": "FZ0T3tDxqgr7jAax8IuOKv-j" }, "locale": "en-US", "text": "", "attachments": [ { "contentType": "application/vnd.microsoft.card.adaptive", "content": { "type": "AdaptiveCard", "version": "1.0", "body": [ { "type": "Image", "id": "Image", "size": "stretch", "url": "https://picsum.photos/id/1033/200/200" } ] } } ], "entities": [], "channelData": { "clientActivityID": "15569132054630.9267z69pgjf" }, "replyToId": "FZ0T3tDxqgr7jAax8IuOKv-j|0000002" } ], "watermark": "4" }

Second Card
{ "activities": [ { "type": "message", "id": "FZ0T3tDxqgr7jAax8IuOKv-j|0000006", "timestamp": "2019-05-03T19:53:30.7467866Z", "channelId": "directline", "from": { "id": "askflowbizzybot", "name": "Ask SPTechCon" }, "conversation": { "id": "FZ0T3tDxqgr7jAax8IuOKv-j" }, "locale": "en-US", "text": "", "attachments": [ { "contentType": "application/vnd.microsoft.card.adaptive", "content": { "type": "AdaptiveCard", "version": "1.0", "body": [ { "type": "Image", "id": "HeroImage", "size": "large", "url": "https://devadmin.atbot.io/images/noimage.png", "horizontalAlignment": "center" }, { "type": "TextBlock", "id": "Product Name", "size": "large", "weight": "bolder", "text": "This is a card for jooo" }, { "type": "TextBlock", "id": "Product Summary", "text": "Product summary or description" }, { "type": "FactSet", "id": "ProductFacts", "facts": [ { "title": "Color", "value": "Red" }, { "title": "Weight", "value": "3" }, { "title": "Brand", "value": "Puma" }, { "title": "Release Date", "value": "2019-02-02" } ], "separator": true } ], "actions": [ { "type": "Action.Submit", "id": "WhoMakesIt", "data": { "keyword": "wfh" }, "title": "Who Makes This" }, { "type": "Action.Submit", "id": "LocateProduct", "title": "Locate Product" } ] } } ], "entities": [], "channelData": { "clientActivityID": "15569132054630.9267z69pgjf" }, "replyToId": "FZ0T3tDxqgr7jAax8IuOKv-j|0000002" } ], "watermark": "6" }

[compulim]

Oh, you know Fiddler? That's great.

The other way, you can also send us the stream of activities coming thru. BTW, the network calls could be sensitive, so please scrap personal information if you do.

[compulim]

Got it, will work on it. Ha, we actually have the same idea on how to debug. 🤩

[compulim]

I bet I got it.

In the activity, please remove channelData.clientActivityID. When the bot send an activity, it should not have this one set, unless the bot specifically put it in.

The clientActivityID appears in the message from the user (from Web Chat), it is an ID that Web Chat used to track/acknowledge if the activity successfully reach the bot or not, a.k.a. "our reference ID".

Web Chat send "message A" to Direct Line channel with this clientActivityID. Shortly after the bot response, Direct Line will send "message A" back with the same clientActivityID. When Web Chat receive this second "message A", it will remove the previous one with same clientActivityID, and replace it with this one.

So, if you send two activities with same clientActivityID, they will get replaced. And since this is a mechanism for message originated from user, bot shouldn't send it. So it's safe to remove it.

[jherres]

Ahh, thanks for the second set of eyes. We confirmed that this is the fix and will make that update on our side.