Botframework-sdk: Conversation state storage questions

[srozga]

Where does the conversation state get stored? i know this is a vague question but I am trying to get a better idea of where the conversation state is stored, what exactly gets stored and for how long. I think some of these questions may have been answered around the docs but our team is having a hard time grasping.

1. Assume we have an EchoStateDialog. We start a conversation with it. Keep emulator open. Rebuild the dialog so that it increments by 2 each time. Run the bot. If we reuse the old conversation, the old EchoStateDialog is still active. It's almost as if a ghost is still alive. I assume that this is because the entire dialog is serialized and stored? Does this gets stored using the default IBotState that goes to state.botframework.com?
2. How is the dialog stored? Is it binary serialization? Is it encrypted at rest? How long is it stored for?
3. Does anything store the conversation history, the messages coming from either user or not? The connector or emulator? Same questions as above. Encrypted? For how long is it stored?
4. Assume we don't want to store ANY conversation data outside our walls. We want to guarantee we control where everything is stored, is it sufficient to provide a custom IBotState?

Thanks!

[willportnoy]

I added an entry to the technical faq to answer your questions:

https://docs.botframework.com/en-us/technical-faq/#where-is-conversation-state-stored

Conversation/message history is more complicated. The bot framework doesn't store messages permanently (other than queuing throughout the system). But LUIS might store utterances to support its "closed loop" of active learning (https://www.luis.ai/Help/#UsingActiveLearning) for example.

[srozga]

Thanks! Very helpful. Could you please provide some clarity on point #1?

[willportnoy]

Yes, the state of the dialog stack is stored in the bot data conversation property bag. The idea is that users can return to a conversation far in the future and the bot's code will be waiting for them, still in the same position in the dialog. It's preserved across code changes.

More details:

https://docs.botframework.com/en-us/technical-faq/#how-do-i-version-the-bot-data-stored-through-the-state-api

[srozga]

Very helpful again! Thanks!

[letmaik]

This is quite helpful but the description for the Bot Framework Emulator does not match the actual behaviour I'm seeing:

The Bot Framework Emulator provides an in-memory implementation of this interface for debugging your bot. This data expires when the emulator process exits.

When I restart the bot application and use the emulator then I am still in the same dialog step. This means that it is not using the in-memory storage. How is this possible? Where is it storing the data?

[willportnoy]

@letmaik Perhaps you're using some combination of ngrok and a deployed bot? Are you using the Node version of the SDK? I'm not sure of its logic for determining the state service uri - you may want to open a new top level GitHub item for discussion.

[letmaik]

No, I don't use ngrok and the bot is not deployed. I use Node. I will open a new issue then.

[snsrkrishna]

Thank you @willportnoy