Botframework-solutions: Inconsistent bot response in Teams channel

[SreekanthOAuth]

• When we say 'hi', it should reply the welcome card as per our flow (which is working fine in Webchat channel). But it's giving different cards and seems this is getting cached.

Need your assistance on this.

Please let me know if you need more details on this.

Thanks,
Sreekanth

[sgellock]

@Virtual-Josh at the very least we're probably going to need to make sure customer understand how to use Sidecar debugging in the Emulator so they can test against the teams environment.

[Virtual-Josh]

Hi @SreekanthOAuth,

There's a couple of things to know.

First, you can use the Bot Framework Emulator to debug your bot within Teams. In the Emulator you can go to "Help"-> "Get started with channels (Bot Inspector)" to see the instructions for how to set it up.

Second, the Emulator will work as documented within a one-to-one and group chat. If you are having issues with that then let us know.

Third, there's a bug in messages in a team. When you do the /inspect GUID command in a team you must respond to the thread that has the /inspect GUID if you want to see the trace activities in the Emulator. Currently the logic in the InspectionMiddleware doesn't handle activities coming from a Teams team chat. I've opened issues on our various repos to resolve the issue. You can view those issues here:

https://github.com/microsoft/botbuilder-dotnet/issues/2955
https://github.com/microsoft/botbuilder-python/issues/428
https://github.com/microsoft/botbuilder-java/issues/142
https://github.com/microsoft/botbuilder-js/issues/1406

Let me know if you have any other questions.

[SreekanthOAuth]

Hi @Virtual-Josh, we don't see any issue in Emulator. Here Bot is working as expected.
The only place where we are seeing the issue is in Teams Channel.

Thanks,
Sreekanth

[Virtual-Josh]

I need you to give me a full overview of what you're doing in order to figure out what's wrong. Here are some initial questions I'm particularly interested in.

1. Are you running this bot in a 1:1 chat, group chat, or team chat?
2. How did you install the bot in Teams?
3. Are you using one of our Samples, or is this your own code?

Alternatively you can view a sample we have for Teams (C# JS) if you want to try to self-service.

[EricDahlvang]

Hi @SreekanthOAuth

Teams Bots will receive a conversationupdate event when the user first adds the bot for 1:1 chat, or the bot is added to a team. Not when new conversations are created in the team or a team member messages the bot. If the welcome message is sent from ConversationUpdate's OnMemberAdded: it will not fire in Teams at the same times as it does in the emulator.

WebChat and the Emulator send a conversation update when they connect (since they both create new conversations during startup).

You could track who has messaged the bot, and respond with a welcome message on first contact. This mechanism isn't built into the sdk, so it is something custom that would need to be coded in your bot.

[SreekanthOAuth]

@EricDahlvang - Sorry for the late reply.

Ok. Let me give some more details about it.

We have Virtual Assistant and custom skills. When a user request for something, the flow goes from Virtual Assistant to respective skill. Inside skill, there is dc.endDialog() method in complete(). This function is getting executing as expected in Emulator, Webchat and SMS channels. But based on our analysis, this seems not working as expected in teams. The context is not being cleared out. So that we see the same skill menu which we have requested first time on every time (even you requested different intent)

Created custom skill (of type Typescript), based below documentation,
https://microsoft.github.io/botframework-solutions/skills/tutorials/create-skill/typescript/3-create-your-skill/

And one more thing is,

• The MSTeams channel was working fine until we move to Virtual Assistant and Skill Routing. Before, Virtual Assistant used to handle all functionalities (based on intent show the dialogue) inside VA itself. So presuming this is major change to facing this issue.

Please suggest on this.

Thanks,
Sreekanth

[amistry79]

I see this issue which discuss similar issue and it workaround in C#bot however we need workaround for typescript bot.

https://github.com/microsoft/botframework-sdk/issues/4774

[Virtual-Josh]

Hey @gabog since we are now talking about skills can you take a look at this ticket to help resolve?

[Virtual-Josh]

Hey @SreekanthOAuth,

I'm going to move this issue over to the botframework-solutions repo since the core problem is the VA/Skills routing.

[lauren-mills]

@SreekanthOAuth - We have updated our VA template and libs to support intro message and auth in Teams. Please try updating your project and let us know if the issues are resolved.

[SreekanthOAuth]

@lauren-mills - Thanks for the update.

We have been working on Virtual Assistant since from 5 months. So we have a lot of updates to our VA.
It would be a great help if you can give details on what are the major changes happened to make it work.

We are on Typescript

Thanks,
Sreekanth

[lauren-mills]

The updates have been made to the C# template, and should be coming to the typescript template soon. The primary changes were to inherit from the TeamsActivityHandler instead of IBot in the DialogBot class. Here is the file in the C# template, and I believe the TypeScript changes should be the same, but let me know if you run into any issues. https://github.com/microsoft/botframework-solutions/blob/master/templates/Virtual-Assistant-Template/csharp/Sample/VirtualAssistantSample/Bots/DefaultActivityHandler.cs

[SreekanthOAuth]

Thanks for the update. We will try and get back to you.

Thanks,
Sreekanth

[SreekanthOAuth]

We see the DialogBot is extending from ActivityHandler in current virtual assistant.

https://github.com/microsoft/botframework-solutions/blob/master/templates/Virtual-Assistant-Template/typescript/samples/sample-assistant/src/bots/dialogBot.ts

We don't find 'TeamsActivityHandler' in 'botbuilder' npm package.

And, below are few more queries
Is this change only for Virtual Assistant or even for skill as well?
This change is not specific to Teams channel, right? If not, how does this impact/works on other channels?

Please let me know if you need any further details on it.

Thanks,
Sreekanth

[lauren-mills]

I can see the teamsActivityHandler in the source for botbuilder-js here. Can you make sure you're on version 4.6?

This change should be made in any bot that will be directly accessed via teams. If you plan on exposing your skill directly (rather than through a VA) you would also need this change to enable Teams activity handling. Otherwise, it is not critical to update the skill project.

Also, despite the name, this activity handler will work for all the other channels in addition to Teams. It just adds additional Teams handling.

[SreekanthOAuth]

After installing the latest version, seeing exceptions while starting the server. Could you please suggest us on this.
errors.txt

[lauren-mills]

@Batta32 - can you take a look at the error log above and give any advice?

[Batta32]

Hi @SreekanthOAuth, we came up with some questions related to the installation you made.

1. Can you verify that you installed the following version?
   a. botbuilder libraries - 4.6.2
   b. botbuilder-skills - 4.4.9
   c. botbuilder-solutions - 4.4.9
2. Have you removed the package-lock.json file and the node_modules folder before the installation? If not, please remove them, update the libraries version to the latest and execute the installation again.

The issue might be related to a disparity of botbuilder version inside the Virtual Assistant against to the ones of botbuilder-solutions and botbuilder-skills.

• [email protected] uses [email protected]
• [email protected] uses [email protected]
• [email protected] uses [email protected]
• [email protected] uses [email protected]

We will be attentive to your answer 😊.

[SreekanthOAuth]

@Batta32 , Thanks for details.

Today, we have tried these and not seeing errors while starting the server. But we are seeing an issue when we request the utterance in teams channel,

And getting staus code as 501 which means NOT_IMPLEMENTED

Error: at DefaultAdapter.<anonymous> (at DefaultAdapter.<anonymous> (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:640:27)at DefaultAdapter.<anonymous> (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:640:27): D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.jsat DefaultAdapter.<anonymous> (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:640:27): 640) at fulfilled (at fulfilled (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:11:58)at fulfilled (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:11:58): D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.jsat fulfilled (D:\home\site\wwwroot\node_modules\botbuilder\lib\botFrameworkAdapter.js:11:58): 11) at process._tickCallback (at process._tickCallback (internal/process/next_tick.js:68:7)at process._tickCallback (internal/process/next_tick.js:68:7): internal/process/next_tick.jsat process._tickCallback (internal/process/next_tick.js:68:7): 68)

Based on these logs and our analysis, seems the Activity Type is going as Invoke and inside below code, it's getting failed.

processActivity(req, res, logic) { return __awaiter(this, void 0, void 0, function* () { let body; let status; let processError; try { // Parse body of request status = 400; const request = yield parseRequest(req); // Authenticate the incoming request status = 401; const authHeader = req.headers.authorization || req.headers.Authorization || ''; yield this.authenticateRequest(request, authHeader); // Process received activity status = 500; const context = this.createContext(request); context.turnState.set(botbuilder_core_1.BotCallbackHandlerKey, logic); yield this.runMiddleware(context, logic); // Retrieve cached invoke response. if (request.type === botbuilder_core_1.ActivityTypes.Invoke) { const invokeResponse = context.turnState.get(exports.INVOKE_RESPONSE_KEY); **if (invokeResponse && invokeResponse.value) {** const value = invokeResponse.value; status = value.status; body = value.body; } **else { status = 501; }** } else { status = 200; } } catch (err) { // Catch the error to try and throw the stacktrace out of processActivity() processError = err; body = err.toString(); } // Return status res.status(status); if (body) { res.send(body); } res.end(); // Check for an error if (status >= 400) { if (processError && processError.stack) { throw new Error(BotFrameworkAdapter.processActivity(): ${status} ERROR\n ${processError.stack}); } else { throw new Error(BotFrameworkAdapter.processActivity(): ${status} ERROR); } } }); }

Could you please suggest us on this.

Thanks,
Sreekanth

[Batta32]

Hi @SreekanthOAuth, we came up with some questions to figure out what's wrong.

1. Can you share the entire log file with the errors if it's available?
2. Can you verify if this issue happens using the Emulator?
3. Does this happens for every intent that you want to execute or for a specific one? (e.g an intent which requires authentication access)
4. Is your dialogBot extending to ActivityHandler or TeamsActivityHandler?

[SreekanthOAuth]

Hi @Batta32 , please find the answers below

1.Can you share the entire log file with the errors if it's available?
_In log stream seeing below response_

2.Can you verify if this issue happens using the Emulator?
_Yes, it's working fine in Emulator and Webchat channels_
3.Does this happens for every intent that you want to execute or for a specific one? (e.g an intent which requires authentication access)
_The first step in our bot is Sign in. Since it's getting failed first step not sure whether this case is same for all or not. But we will get back to you on this._
4.Is your dialogBot extending to ActivityHandler or TeamsActivityHandler?
_It's extending with TeamsActivityHandler_

[Batta32]

Hi @SreekanthOAuth, sorry for the delay. We managed to reproduce your scenario which we solved overriding the method handleTeamsSigninVerifyState on the DialogBot which extends TeamsActivityHandler. As you can see in TeamsActivityHandler, it contains some methods that are not implemented, which we should override.

With this said, could you tell us if you are implementing the method handleTeamsSigninVerifyState in the class that implements TeamsActivityHandler? If not, you can use the code of below:

public async handleTeamsSigninVerifyState(context: TurnContext, state: SigninStateVerificationQuery): Promise<void> {
    await this.dialog.run(context, this.state);
}

To accomplish this, we followed these steps using the Virtual Assistant Sample:

1. Change DialogBot.ts to extend TeamsActivityHandler
2. On DialogBot, implement the method handleTeamsSigninVerifyState overriden as shown in the Teams Auth Sample
3. Run the Virtual Assistant Sample in Teams and make sure that it is receiving the request type Invoke in botFrameworkAdapter.
   Note: This was important as the type Invoke is received only when logging in from a Teams channel.

The overridden handleTeamsSigninVerifyState is called after signing in in Teams, instead of await bot.run(turnContext); found in index.ts.
If the method is not implemented, it would default to use the server post in the index, and it causes the context.turnState.get() in botFrameworkAdapter to return an invokeResponse with status 501, which in turn is used to create and send a response with said status.

Additional Context

• NodeJS Sample #46: Teams Auth
• NodeJS Sample #57: Teans Conversation Bot
• Add authentication to your bot via Azure Bot Service
  
  
  
  • Virtual Assistant: Extend to Microsoft Teams
  
  

_Scenario reproduced_

We hope this helps you 😊.

[SreekanthOAuth]

Hi @Batta32 , thanks for the update.

I have tried to adding these changes but getting some errors

After adding changes, dialogBot.ts file

`

import {
ActivityHandler,
BotTelemetryClient,
ConversationState,
EndOfConversationCodes,
Severity,
SigninStateVerificationQuery,
TeamsActivityHandler,
TurnContext,
StatePropertyAccessor} from 'botbuilder';
import {
Dialog,
DialogContext,
DialogSet,
DialogState,
DialogTurnResult } from 'botbuilder-dialogs';

export class DialogBot extends TeamsActivityHandler {
private readonly telemetryClient: BotTelemetryClient;
private readonly solutionName: string = 'Ecolab Virtual Assistance';
private readonly rootDialogId: string;
private readonly dialogs: DialogSet;
private readonly dialog: Dialog;
private readonly dialogState: StatePropertyAccessor;

constructor(
    conversationState: ConversationState,
    telemetryClient: BotTelemetryClient,
    dialog: T) {
    super();

    this.rootDialogId = dialog.id;
    this.telemetryClient = telemetryClient;
    this.dialogState = conversationState.createProperty<DialogState>(this.solutionName); // new one
    this.dialogs = new DialogSet(this.dialogState);
    this.dialogs.add(dialog);
    this.onTurn(this.turn.bind(this));
    this.dialog = dialog; // new one
}

//tslint:disable-next-line: no-any
public async turn(turnContext: TurnContext, next: () => Promise<void>): Promise<any> {
    // Client notifying this bot took to long to respond (timed out)
    if (turnContext.activity.code === EndOfConversationCodes.BotTimedOut) {
        this.telemetryClient.trackTrace({
            message: `Timeout in ${ turnContext.activity.channelId } channel: Bot took too long to respond`,
            severityLevel: Severity.Information
        });

        return;
    }

    const dc: DialogContext = await this.dialogs.createContext(turnContext);

    if (dc.activeDialog !== undefined) {
        const result: DialogTurnResult = await dc.continueDialog();
    } else {
        await dc.beginDialog(this.rootDialogId);
    }

    await next();
}

public async handleTeamsSigninVerifyState(context: TurnContext, state: SigninStateVerificationQuery): Promise<void> {
    await this.dialog.run(context, this.dialogState); // getting error here.
}

}

`

Here w.r.t dialog and dialogState create new attributes.
In this.dialog there is no run().

Could you please suggest.

Thanks,
Sreekanth

[Batta32]

Hi @SreekanthOAuth! If you are not able to use the run method, you should add a type for T which contains that method, as follows:

1. Update DialogBot class as DialogBot<T extends MainDialog> extends TeamsActivityHandler or directly specify a concrete type for dialog as dialog: MainDialog.
2. Implement the run method in MainDialog.

You can check those changes in our pushed branch with a modified Virtual Assistant Sample that implements the Teams Sign In.

Finally, you can take a look at the NodeJS Sample 46 for how it implements the run method with waterfall dialog.

Let us know if this helps or you have any other issue 😊.

[lauren-mills]

Closing due to inactivity. Please reopen as needed.

[SreekanthOAuth]

@lauren-mills , could you please reopen the issue as the issue still exists.

We have tried your suggestions based on recent comments. However, we are still encountering with the problem.

The real issue over here is,

• MS team channel doesn't support EndOfConversation activity type which is returned from the skill bot.

Please do let us know if you need any additional information.

Thanks,
Sreekanth From Ecolab.

[VictorGrycuk]

Hi @SreekanthOAuth!

We are looking into this.

[VictorGrycuk]

Hi @SreekanthOAuth,

We were able to reproduce your issue using the master branch, but also made the same test on next and it worked without issues.

These are the steps we followed to reproduce the issue:

1. Deploy a Sample Virtual Assistant
2. Deploy a Sample Skill
3. Create a Sample Skill, but make sure to use different utterances, questions, and answers to differentiate it from the first skills, such as:
   
   
   
   • SampleResponses.json
   
   
   • sampleSkill.lu
   
   
4. After deploying the Virtual Assistant and the Skills, connect both Skills to the Virtual Assistant using the command botskills connect --remoteManifest "http://localhost:<SKILL_PORT>/api/skill/manifest" --luisFolder "<PATH_TO_LUIS_FOLDER>" --ts
5. Start the Virtual Assistant, and run ngrok using the same port used to run the Virtual Assistant
   
6. In Azure Portal, go to the Virtual Assistant bot resource, Settings > Messaging Endpoint and configure the ngrok URL
   
7. Create a manifest.json of the Virtual Assistant to use it on Teams following these steps
   
   
   
   • Alternatively, use the manifest of the sample Teams Conversation Bot as template, along with its icons
   
   
   • Make sure to replace <<YOUR-MICROSOFT-APP-ID>> with the appID of the Virtual Assistant
   
   
8. Zip the manifest.json and the icon files, and import it to Teams
   
9. Initialize the Virtual Assistant application and send the utterance of the first Sample Skill
10. After finishing with the waterfall dialog of the Sample Skill, an exception will be thrown:
    
11. The exception is thrown in the method deserializeResponseBody() of the file deserializationPolicy in ms-rest-js:
    

We also did the same test using the next branch, and the issue was not found

As it can be seen in the image above, after finishing the waterfall dialog, the first Skill Sample hands off the control to the Virtual Assistant, then the utterance for the second Skill Sample is sent (alternative) and it is executed successfully, as noted by the "What else can I help you with?" sent by the Virtual Assistant.

Note: To use the next branch, it is necessary to build the BotBuilder Solutions and BotBuilder Skills locally, and update the package.json of both the Virtual Assistant and Skill to use the local .tgz files, as they are not released yet.

This means that this issue will be fixed in the soon to be released version of TypeScript library.

[SreekanthOAuth]

Sorry for the late reply. I'll check and give an update.

[VictorGrycuk]

Hi @SreekanthOAuth,
The next branch has been merged with the master branch.

Please test again with the newly implemented changes.

[darrenj]

Please re-open if you run into the issue again

[amistry79]

Please re-open this. We tried to fixed based on the suggestion "replacing" ActivityType.EndOfConversation to ActivityType.HandOff and implementing TeamActivityHander. However, the fundamental issue is EndOfConversation ActivityType is not supported by MSTeam. We are really stuck and can not promise our internal stakeholders for MSTeam bot availability.