Brigade: Brigade Build does not fail for sequential Jobs (unless it's the first Job)

Created on 7 May 2019  路  3Comments  路  Source: brigadecore/brigade

Actual Behavior: Brigade Build always succeeds if first Job succeeds. Tested with both promise chains and groups.

Expected Behavior: Brigade Build should fail if any Job fails

    var group = new Group();

    var job1 = create_job(...);

    group.add(job1);
    var job2 = create_job(...);
    group.add(job2);

    group.runEach().then((res) => {
        console.log('RES: ', res);
    }).catch((err) => {
        console.log('ERR: ', err);
    });

Kashti Dashboard:
Screen Shot 2019-05-07 at 10 50 56 AM
Screen Shot 2019-05-07 at 10 50 34 AM

kubectl get pod ...:
Screen Shot 2019-05-07 at 10 53 26 AM

Output of brig version:
v1.0.0-23-g1b849be

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):
AKS

Most helpful comment

@gfrankel1997 This may be a case of inadequate documentation.

In the example provided above, since errors are caught via the catch statement, the pipeline won't fail unless we explicitly tell it to (say, via a throw). This design enables finer control over when and how a pipeline might fail. For instance, we might add retry or other logic in the aforementioned catch section to handle acceptable/expected job errors instead of failing the entire pipeline.

In this case, if we want the pipeline to fail, we can either throw the error we catch or remove the catch stanza entirely, thus enabling it to bubble up all the way to the worker to log and fail the build appropriately.

All 3 comments

@gfrankel1997 This may be a case of inadequate documentation.

In the example provided above, since errors are caught via the catch statement, the pipeline won't fail unless we explicitly tell it to (say, via a throw). This design enables finer control over when and how a pipeline might fail. For instance, we might add retry or other logic in the aforementioned catch section to handle acceptable/expected job errors instead of failing the entire pipeline.

In this case, if we want the pipeline to fail, we can either throw the error we catch or remove the catch stanza entirely, thus enabling it to bubble up all the way to the worker to log and fail the build appropriately.

That being said, this might also open up the discussion for adding a custom field to the Build object passed between the API and various front-ends (brig, brigadeterm, Kashti) that users might be able to set from brigade.js.

@vdice @radu-matei Imagine that.. catching the error actually catches it. That design is perfect, lets me catch the error, log it wherever, retry logic, etc. and then have it also bubble up to the Build level. Thanks for the clarification guys. Closing.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

danmx picture danmx  路  7Comments

mattfarina picture mattfarina  路  4Comments

radu-matei picture radu-matei  路  6Comments

LittleColin picture LittleColin  路  7Comments

krancour picture krancour  路  5Comments