Conda-forge.github.io: Adding a status page

[jakirkham]

After having some issues yesterday during deployment (due to a package having a . in its name) and thinking back to the fact that this problem has occurred before for us while trying to explain this to a user that expected everything worked, I was thinking maybe we would benefit from having a status page. This way we could transparently communicate the status of different parts of our pipeline. Additionally, it would be nice if it could be combined with our automation to announce issues.

Given how much we :heart: repos here, one possibility that I noticed was a status page that used a repo ( https://github.com/pyupio/statuspage ). We already are quite familiar with the process of automating interactions with repos. Errors in feedstock generation could be filed as issues with one label. Errors in webpage deployment could go under another label. Errors updating the feedstocks submodules could go under yet another label (thus solving this issue https://github.com/conda-forge/conda-forge.github.io/issues/70). As labels are required for an issue to show up in the status, we don't need to worry about users opening issues against this status repo and accidentally affecting the status. This would also do double duty as we could be notified as these problems occur letting us more quickly resolve them. Also, it would provide this information to our users without distracting us from trying to get out a quick fix. Once we resolve an issue, we can close it, which will update the status. Alternatively, we can automate the closing of issues too. This won't leave us dependent on yet another service to get the job done. It will just rely on GitHub and whatever CIs are relevant.

We can certainly consider other ways to do this. Right now, I think this idea sounds like the easiest to setup and will nicely integrate with our existing workflows.

[pelson]

Given how much we :heart: repos here, one possibility that I noticed was a status page that used a repo

Hehe. We don't love repos for the sake of loving repos - we love them because they were the right abstraction for conda recipes :smile:

That said, it is a pretty neat idea! One question though: what are the kinds of statuses we've seen recently, and would having the status page have improved the user experience? I'm reluctant to implement a status page if we are gradually refining away all of the issues anyway.

[jakirkham]

Sorry for the long hiatus on this one. I got really excited about and then went back to dealing with the problems at hand.

Hehe. We don't love repos for the sake of loving repos - we love them because they were the right abstraction for conda recipes :smile:

I know. It was a bit of a joke. That being said, I agree with the present model.

That said, it is a pretty neat idea!

Thanks. 😄

One question though: what are the kinds of statuses we've seen recently, and would having the status page have improved the user experience?

Here is an example issue (rate limiting) ( https://github.com/conda-forge/staged-recipes/issues/489 ) that I find myself opening periodically. Though rate limiting is a global not local issue. At the time, one of the issues was empty feedstocks ( https://github.com/conda-forge/conda-forge.github.io/issues/70 ). Some of this would be helpful from the management side like letting us know some things weren't being converted to feedstocks. Though this can also be of value to users that thought their recipe(s) were converted to feedstock(s), but failed in actuality. This way if they are adding things that depend on it (was happening in the particular instance that motivated this issue), the user can be made aware of the problem in this way. Other ideas might be having a common notification ground about connected service issues (e.g. CircleCI, Travis CI, AppVeyor, Heroku, GitHub, DockerHub, and possibly others in the future).

I'm reluctant to implement a status page if we are gradually refining away all of the issues anyway.

In some cases, I find we have these issues at various places (e.g. webpage repo, staged recipes, maybe even feedstocks, in the future the docker-images repo). By having a singular status page repo, we would actually be able to consolidate these issues instead of cross referencing across repos. Further it would avoid noise that occurs by having these operational issues intermingled with topic driven issues like the organizational ones we have here, package requests on staged recipes, etc.. It would also avoid having to ping each other and draw attention to each other all over the place. By having a singular place to raise these issues, we could also see existing issues related to this problem and discuss any strategy as needed. I can think of lots of nifty uses cases for this, but will stop there.

[pelson]

Sold. What do we need to do to make it happen?

[jakirkham]

I think we need a repo, a new token (possibly bot user?), and probably use another thread on Heroku.

[pelson]

I think we need a repo, a new token (possibly bot user?), and probably use another thread on Heroku.

You have the power on all of these. My advice first though - get it going as you and then we can migrate it to an appropriate user (once we know the load on GH etc.).

Are you happy to run with this?

[jakirkham]

cc-ing @ocefpaf so that this is on your radar at least as something we are planning to do.

[jakirkham]

Are you happy to run with this?

Yes. Will get back with you when I have something more concrete.

[jakirkham]

cc-ing @astrofrog so you can see what I have on mind for this status page. Also, if you have some feedback on this, please let me know.

[jakirkham]

I am creating this repo ( https://github.com/conda-forge/conda-forge-statuspage ) for this purpose.

[jakirkham]

Decided to renamed to ( https://github.com/conda-forge/status ). The main reason being that someone can visit ( https://conda-forge.github.io/status ) to get the status. Though happy to discuss this with others here.

[jakirkham]

Copying you guys here as you are watching the repo currently, @ocefpaf @janschulz.

[jakirkham]

Adding the dependencies in this PR ( https://github.com/conda-forge/staged-recipes/pull/775 ).

[pelson]

Are interested in moving this forward without the dependencies on conda-forge so that we could potentially parallelise progress?

[jakirkham]

Well, the dependencies are ready to go at this point. They just need a review/merge. If there are formatting changes required, then I agree someone else could/should be welcomed to take them over. Alternatively, those sorts of changes could be saved for feedstocks depending on how people feel.

The main issue I have these days is really just having enough time. I say that while fully acknowledging having a status page will be a real time saver once it is up.

Though I agree this is very important to get moving. Hence why it has gotten some more attention of late. 😉

Will start playing with phase 2 soon.

[jakirkham]

So, I have generated the status page. Though still needs more work before it is operational.

After going through that process, I'm not sure that going with the PR deps as is will be the best move. This may take a bit more customization than that. I'll work on it a bit and see what I can come up with.

[jakirkham]

Need to add a patch to out pygithub package to proceed. Here is the PR ( https://github.com/conda-forge/pygithub-feedstock/pull/9 ) for adding that patch. Details in that PR. A review/merge of that change would be greatly appreciated.

[jakirkham]

Also, we need to make the certificates easier to discover. I found setting some environment variables will do the trick. Please see this PR ( https://github.com/conda-forge/ca-certificates-feedstock/pull/2 ) for details on how I propose we do this.

[jakirkham]

To ensure the change to ca-certificates ( https://github.com/conda-forge/ca-certificates-feedstock/pull/2 ) is honored on Heroku. We need to add proper environment activation, which this PR ( https://github.com/pl31/heroku-buildpack-conda/pull/2 ) should do.

[jakirkham]

Are interested in moving this forward without the dependencies on conda-forge so that we could potentially parallelise progress?

So, I have tried to do what I can as far as setting up the page. I think we will have to chat about how we want to configure this with Heroku, @pelson. Not really sure how things are working now or how to insert this into the mix. That being said, what we actually need to do is quite simple. We just need to pickup a webhook any time an issue is opened on that status page and run one command when that happens.

Though honestly the hardest part of this process has always been the dependencies. 😞 The problem with trying to do this without them is we will end up having to write our own buildpack from scratch and do lots of weird development installs, which sounded like more of a pain. That being said, all of the changes shown above work for me and do the job (excepting Heroku as it is untested yet).

[jakirkham]

This has been implemented with PR ( https://github.com/conda-forge/conda-forge-webservices/pull/34 ) and rebuilt on Heroku. Opening issues with a label for the service and the service status on repo ( https://github.com/conda-forge/status ) will result in a payload being sent to Heroku, which will then update the webpage. History will be logged below. We ( @pelson and I ) tested this earlier to make sure it worked in several cases. Please open any service issues encountered on that repo with related info. Thanks @jayfk for all of the help getting statuspage packaged. Also thanks @pelson for helping get this live. Please ask any questions you may have.

cc @conda-forge/core