Probot: Configuring GitHub App scopes with multiple plugins is very complicated :/

[anglinb]

Problem Definition

👋 Let me preface this by saying probot is awesome and I appreciate all the work that has gone into this project!! 😎 However, by far the most ambiguous/frustrating part of getting up an running with probot is setting up the GitHub App & configuring the right webhooks--I frequently resort to just checking all the boxes. 🙃

Current State of Things

I've noticed that most probot plugins provide a /docs/deploy.md file which is largely the same except for the requested permissions. If I want to setup these permissions, I have to manually union all the requested permissions and the actually create my GitHub App. Here are some examples of these files: stale, settings, autoresponder.

Possible Solutions

An optimal solution, in my mind, would be an GitHub App creation endpoint. However, this is out of my hands and you would still need to seed the use of such an endpoint with some sort of authentication.

The next best solution (that would also prepare probot for any feature like an app creation endpoint) would be to expose a method the robot class that would allow plugins to define their requested scopes. This could be used in a "postinstall" step on npm install. This could generate a text output that shows you how to configure the GitHub App or even generate HTML that look like the create app page so you can just mindlessly copy all the settings.

I'm happy to open a PR for this if @bkeepers and @hiimbex agree that this could be useful. Lmk your thoughts!

[hiimbex]

We were talking earlier today about making the redirect link in the app settings direct to for setup:

You get directed here after you've installed the bot. _However_, when you install the bot as a deployed app, the permissions are automatically factoring in for a user to install it and all they have to do is accept, not recreate the steps of the developer. The only time you would need to manually do that is if you were trying to run the bot locally. But it is a pain point. For development purposes, I use one github app and assign it all the privileges (h/t @bkeepers)

But I definitely see this as having other potential set up use cases! All these workflows are semi-awkward though, so I'm not sure the best way of addressing the true problem. 🤔

[anglinb]

Chatted w/ @hiimbex wanted to capture a few of the take aways:

Currently most probot plugins are hosted on their own and available to GitHub users.

This means there is not a _huge_ use case end users needing to create their own GitHub App unless during development. (But in dev you can just check all the boxes ¯_(ツ)_/¯)

There are only a few plugins that are just a combination of other plugins--most probot plugins are hosted on their own. Even combinations of plugins are usually hosted already

Using behaviorbot/welcome as an example, it is a combination of multiple plugins but again it is hosted for anyone to use and it's only a one-time cost of setting up a GitHub app with permissions and it is also hosted.

Overall, the use case of hosting your own collection of probot plugins might not be a super huge use case and might be a bit tangential to the way GitHub Apps were intended to be used.

Might still be useful to consider probot plugins that are built by the community. It puts the burden on the developer to pay for/find sponsors for hosting their app or force end users to host it themselves and go through the setup issues we've brought up. Seems like most probot plugins are currently hosted by the probot org so this issue might have jumped the gun a bit/mis-understood the current use case.

Anyways, closing this for now. Thanks @hiimbex for taking the time to talk through this!! ✨

[bkeepers]

An optimal solution, in my mind, would be an GitHub App creation endpoint.

@anglinb I had the exact same thought, and have talked with @tarebyte about it. Creating new GitHub Apps is not necessarily done very often, but I think offering a programatic way to set the permissions would benefit most GitHub Apps.

I wonder if there's a way we could simplify this with documentation in the mean time, either on https://developer.github.com/ or in Probot.

cc @kytrinyx

[anglinb]

@bkeepers I agree--would like to see an app creation & update endpoint if possible. To simplify my reasoning, I think the single metric: Time from "Wanting to play with a GitHub App" to "A webhook payload printing in my terminal" is a really good metric for "ease of getting started".

Imagine if probot provided a command to bootstrap your GitHub App, setup your localtunnel, and set the webhook endpoint to point to your localtunnel domain.

A shining example of this experience in practice is the GraphQL Explorer: The time to go from "I want to try GitHub API v4" to "first response to a GraphQL query" is less than a minute, maybe single digit seconds. I think this can make a huge difference in the perception of the difficulty in starting GraphQL.

I totally agree that documentation is a great way to get started with this w/ a minimal amount of effort. I'll put some thought into how we could improve both Probot & developer.github.com docs and open issues in the relevant repos. Feel free to create those yourself if you already have some formed thoughts and I can just add on 👍