Allennlp: [Proposal] Refactor and move CI to GitHub Actions

[epwalsh]

So far I've had a couple discussions about refactoring and moving our CI to GitHub Actions (or another alternative) with @schmmd and @dirkgr, and I want to see if anyone else has opinions on this.

IMO the downsides of the current system on TeamCity are:

• The builds take forever. This is not necessarily an issue with TeamCity, but definitely suggests we need to refactor. This could be solved by better caching, running more things in parallel, and also removing or improving the handful of unit tests that take a long time.
• It's not scalable.
• The CI configuration is not source controlled and is hidden from outside contributors.

I think all of these issues could be addressed using GitHub Actions. We could also run tests on different operating systems (Linux, Windows, and Mac OS X all supported) in parallel.

One potential downside is that we may have to set up our own runner for the GPU tests.

[schmmd]

While I'm completely open to GitHub actions, I think the slow build speeds is orthogonal to our build system. TeamCity should cache effectively, but our tests are slow and things don't run in parallel.

I would add to your list that we need to manage auth and infrastructure for TeamCity.

[nelson-liu]

The CI configuration is not source controlled and is hidden from outside contributors.

I think this is huge, and the main reason why I'd like to see a switch (as a mostly-outside collaborator).

[epwalsh]

I would add to your list that we need to manage auth and infrastructure for TeamCity.

I guess we'd still have to do some management if we need any self-hosted runners. But it sounds like it wouldn't be as involved as managing TeamCity.

[epwalsh]

The first steps to this are now in this PR: https://github.com/allenai/allennlp/pull/4097

I think the only thing missing is GPU tests since that will involve a self hosted runner.

[bryant1410]

It was already said, but I wanted to emphasize that this is indeed neater for external contributors, yeah.

[epwalsh]

Closing this as the remaining pieces are being tracked by https://github.com/allenai/allennlp/issues/4108 and https://github.com/allenai/allennlp/issues/4096