Botkit: Slack retrying everything, not convinced I'm ever responding quick enough (using localtunnel.me)

[peabnuts123]

Hey, I'm not sure what my issue is here so I'm asking on here for some insight. I'm writing a bot for Slack using the SlackAdapter. It sources its data from a third-party API so it needs to first make a request and then reply on Slack after fetching the data it needs. Now, Slack has some pretty strict guidelines around receiving an acknowledgement from your bot within a 3 second window of their request. As per Issue #134, botkit supposedly automatically ACKs with an empty response, so this shouldn't be much of an issue. However, Slack is constantly retrying almost every event it sends me. Slack notes that if you retry too many requests, your app will eventually be disabled.

Now this is just in my local development environment (I have not deployed this anywhere yet), I'm using localtunnel.me to expose my dev server but I have a fast connection, I can't see why it would be more than a 3000ms round-trip to Slack's servers if the Botkit library is in-fact auto ACK'ing for me as soon as it receives its request. This is kind of a problem as I've had to build a cache for my dev server so that my bot only responds to messages once, and who knows what will happen once I deploy the bot to my server. This can't be common behaviour, does anybody have any insight as to what's the slow link in my setup?

Outline of my code is as-follows:

// Configure botkit
const adapter = new SlackAdapter({
  clientSigningSecret: process.env.clientSigningSecret,
  botToken: process.env.botToken,
  redirectUri: process.env.redirectUri || '',
});
adapter.use(new SlackEventMiddleware());
adapter.use(new SlackMessageTypeMiddleware());

const controller = new Botkit({
  adapter,
});

// Wait for bot to start
controller.ready(() => {
  controller.hears(['.*'], ['direct_message', 'direct_mention', 'mention'], async function (bot, slackMessage) {
    /* Make a request to another API */
    const response = await fetchDataFromThirdPartyAPI(slackMessage.text);

    /* Reply based on the response */
    await bot.reply(slackMessage, `Response from API: ${response.output}`);
  });
});

Context:

• Botkit version: 4.0.2
• Messaging Platform: Slack
• Node version: v10.13.0
• OS: OSX v10.14

[benbrown]

The 4x branch no longer does an immediate ACK - it will wait until your handler finishes.

You could put your bot.reply inside a .then for your API call function and return without waiting for it to complete. However, since that will result in an "out of turn" message, you will have to refresh the bot's internal context by calling await bot.changeContext(message.reference); before using bot.reply as seen here:

https://github.com/howdyai/botkit/blob/master/packages/testbot/features/websocket_features.js#L76

[peabnuts123]

Ah I see! Thanks for the heads up, I guess that works just fine. Is there room now to re-introduce a command like bot.replyAcknowledge, to instruct the Slack adapter to send an ACK back to the server?

i.e.

controller.hears(['.*'], ['direct_message', 'direct_mention', 'mention'], async function (bot, slackMessage) {
  /* ACK to Slack */
  await bot.replyAcknowledge();

  /* Make a request to another API */
  const response = await fetchDataFromThirdPartyAPI(slackMessage.text);

  /* Reply based on the response */
  await bot.reply(slackMessage, `Response from API: ${response.output}`);
});

[benbrown]

replyAcknowledge doesn't really make sense with the new async/await pattern in place for all those handlers. To acknowledge, you have to return from the handler. This allows you to throw an http error code if an error occurs, for example. It is also used by the Slack adapter when responding to slash commands, dialog submits and a few other types of webhook that expect a response.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[MatissJanis]

Can this be fixed? It's very annoying that slack keeps re-sending messages if a await process is taking really long.

[minode-raj]

Can we give an option to avoid all retries?
Slack send X-Slack-Retry-Num option in it's header. Can we have a config where all retries are ignored?

[adamdabbracci]

This continues to be an issue, especially if you make calls to external services like @peabnuts123 is doing.

I've tried writing middleware to solve the problem. I tried with webserver middleware, but it doesn't seem to respond to Slack messages at all. Then I tried sending a bot response like "working on it..." to each message, but that fails if the input wasn't from a chat message (such as a dialog submission). There doesn't seem to be a way to intercept the Botkit message while still having access to the underlying webserver (i.e. to call res.send(200) but continue processing the message). I suppose the only option would be to add a response in the code for _every_ interaction, but that seems like an anti-pattern.

I definitely understand why doing this with async processes doesn't make sense, but neither does Slack's 3 second requirement. I +1 the option of re-adding replyAcknowledge to resolve this problem, as it seems to be a simple solution without introducing complexity.

Thanks :)

Update 2/2/21
Still trying to fix this. I primarily use dialogs, so this might be a unique use case.

I tried handling sending an HTTP Post to the "response_url", but get a 400 or 500 error from Slack, no matter what I pass in the body. I'm not sure if this is an issue with their documentation, or I'm misunderstanding that URL, but there's no additional info and I can't find anything on Google.

I also tried replying with a message then re-setting the context(using the example provided) - the message shows up in the Slack conversation below the dialog, yet it still gives me the "We had some trouble connecting" error after 3 seconds.

Update 2/2/21 Part 2
I finally got it working, but I don't love the solution. I moved all of the long-running workflows into their own class within async functions. From the BotController, I'm invoking those async functions, but not awaiting them. They continue to run in the background even though the handler has completed. This is definitely not a proper solution, but without access to the res object in one of the Botkit middlewares, I can't find a better option. If anyone comes up with a more ideal solution, I would love to see it!

[AdrianSchneider]

The best I could come up with was to ignore slack retries (my use case: we'd rather manually retry than deal with duplicates being handled)

  new BotKit({
    webserver_middlewares: [
      (req: any, res: any, next: any) => {
        if (req.headers['x-slack-retry-num']) {
          res.status(200).send()
          return
        }
        next()
      }
  )