Orchardcore: Webhooks module ideas

[deanmarcussen]

Was starting to implement Webhooks to ping google and bing from the Sitemaps module, but got a bit carried away and started architecting a more generic Webhook module, that other modules could plug into.

Haven't gone far with the code, so just putting this up for some feedback to see if community would be interested in it, and offer opinions about the architecture ideas.

Here's a sample scenario (using Slack and Sitemaps, should extend to cover Discord and any other one way Webhook)

The sitemaps module wants to be able to ping google and bing from the admin.

It would be convenient if the site admin could also configure it (without writing any code) to send (for example) a Slack notification to an admin channel.

So the idea would be that a Webhooks module and Providers or Consumers of Webhooks should be able to do the following

• Describe themselves as Consumers of Webhooks

  • The SitemapSubmissionService should describe itself as a Consumer so the admin can configure any Webhook provider to plug in when sitemaps are submitted
  • A Slack Webhook Provider should be able to use this description to configure different Consumers to use different channels, and enable / disable a Consumer

• Sits below Workflows, so they can be used by other Consumers, but

  • A Slack Workflow Consumer activity should be able to use the Slack Webhook and adjust it's channel (so the admin settings for Slack / Workflows Consumer are more defaults for the Workflow Consumer)

• Providers and Consumers would be features, so enabled or disabled from the module configuration, and then configured from their own settings driver

• Webhooks can lean on the IHttpClientFactory, but not restricted to only use HttpClient

  • Could also use Sendmail Client, or any other Api client

• Limited results would be provided on sending a Webhook, i.e. Webhooks are intended as a simple one way transmission, not intended to be used as an ApiClient which requires a lot of back and forth / negotiation

• Both Consumers and Providers should be extensible

  • Someone could easily add an 'admin' Consumer and listen for events to send admin notifications to Slack or any other Providers
  • Or add a Provider which all the other Webhook consumers could configure start using

Any thoughts?

[jeffolmstead]

This sounds very interesting. In the original Orchard I had developed a sitemap "ping" functionality that could be driven off a content publish event - I love that you are considering a much more generic approach! I don't know all the details but the concept sounds great. Will watch to see what others come back with.

[scleaver]

Sounds amazing!

[deanmarcussen]

@jeffolmstead Thanks. Yes hoping to support a ping of Content Publish as a configurable option as well. Figure auto ping is good for smaller sites, but for larger sites with many publish events a day you should be able to turn it off, and either ping from the admin, or ping once a day from a background task etc.

Autoping will be simple for sites with one sitemap, but not sure what should happen when the Content Item is contained in an index. Do you ping google with the new sitemap-index or the content-sitemap?

[sebastienros]

Do you mean this? https://github.com/OrchardCMS/OrchardCore/pull/2323

Or do you mean to just create new workflow tasks/events that are dedicated to Slack and other providers. We already have generic webhooks that you can configure, but having dedicated ones that are pre-configured and also use global settings (slack api keys) would be useful.

[deanmarcussen]

Yup, #2323 is pretty much what it looked like in my head.
@jrestall great stuff.

Glad I asked, before writing any code - should have searched closed issues as well.

For the use case of sitemaps, on reflection, it totally makes more sense to write a basic workflow and include it as an optional recipe - did not realise that the Signal Event could be used to drive workflows programatically till I read the docs (dup), but using that it could trigger how the site developer wants it to.

After a few conversations about automatic submissions with an seo colleague there's too much variance in how you should / might submit index sitemaps / content sitemaps / individual content items to make sure you don't overrun your google robot crawl budget (apparently a mysterious unknowable google thing) by over submitting, to be able to write something that will work with every scenario/site.

I'll wait for some feedback on #3229 (no rush!) and think about including a basic workflow recipe in it and probably a button on the sitemap nodes that will emit the Signal. (as a basic way of submitting, that isn't automatic).

Re slack and other webhooks, yes it would be useful (api keys / channels etc), and would also look particularly good on the 'out of the box' feature set to support some preconfigured providers.