Lokomotive: Refactor CI pipeline to only build the binary and run unit tests once

[invidian]

This is how our current pipeline flow looks like:

This is sub-optimal as many steps are running multiple times. It also makes some check to fail other important tests (see #769 for more details).

We should make the pipeline look like this:

Which as far as I know, should be quite easy to do with Concourse.

[knrt10]

Really nice work on the charts @invidian. 👍. love them

[invidian]

I think following pipeline should be possible to implement in Concourse:

[invidian]

Okay, so I played around a bit and I think I've implemented it properly:

You can see it here: https://yard.lokomotive-k8s.net/teams/main/pipelines/pull-request

However, I found some issues with the implementation:

• Having only one resource definition of this repository makes it impossible to use ignore_paths feature, so conditional running of e2e tests is currently not implemented. We would have to add additional "gate" task, which would control if the e2e test should run or be skipped.
• Ideally, all tasks which run should share it's Go cache, so only first step is slow (or even first run of first step) and then things should re-use Go build cache. Unfortunately, cache in concourse can only be shared between the invocations of tasks, so very efficient caching is not possible.
• concourse outputs can only be shared between steps in single job, which means that the lokoctl binary must be still build for every e2e test independently.

And here is rest of the findings and results:

• Each pipeline step can now nicely report the status to GitHub, so there is no need to open a build page, but you can just look up in GitHub status which step failed (https://github.com/invidian/lokomotive/pull/2)
• e2e tests depends on unit tests step and unit tests and linter steps depends on build step, so there is better visibility of what failed
• the pipeline definition has now less duplication, however it is still rather verbose
• it turns out, we should be using on_success instead of yet another step for reporting success state to GitHub
• it seems this is proper way of building more complex concourse pipelines and it would definitely be nicer to have just a few pipelines like pull-request, master and release instead of separate pipeline for each provider

[invidian]

Building, unit tests linter etc now runs in GitHub actions, so this is not really needed anymore.