Brigade: bug: observer doesn't notice vcs init container failures

[krancour]

If the worker's vcs init container fails, the worker should be transitioned to a failed phase, but it isn't currently.

Similarly, if a job's vcs init container fails, the job should be transitioned to a failed phase, but it isn't currently.

[vdice]

Let's check this again soon. At least with sad case e2e test added in https://github.com/brigadecore/brigade/pull/1229 (malformed git repo url), the worker appears to fail correctly.

[krancour]

@vdice can I reassign this one to you? As you indicated, this bears double-checking.

[vdice]

I have a branch with updated vcs tests and finer-grained assertions -- that I'll PR regardless -- but I was curious around the following:

Similarly, if a job's vcs init container fails, the job should be transitioned to a failed phase, but it isn't currently.

I can't quite figure out how to test the scenario where a worker's vcs init container succeeds -- as it must if any jobs are to be started -- but then setting things up for the job's vcs init container to fail. This seems currently impossible, as the job's vcs details are always inherited from the worker and don't appear to be overrideable. Or did you have an approach in mind for this scenario?

[krancour]

Good point. Realistically, if a worker's git initializer succeeded, any and all jobs that also use a git initializer should have that initializer succeed, unless.... unless there's a network failure or some such, but I'm not sure how we'd inject such a fault.

Let's just make sure the observer gets everything right when a worker's git initializer fails and then trust that similar logic works for jobs with failed initializers also.

[vdice]

Nothing to do here; https://github.com/brigadecore/brigade/pull/1252 added test verification.