Newman: How do I make Jenkins build fail when Newman tests fail

[Sripathi1983]

eg- I have 2 collections one passes and one fails , both are fired by two separate newman commands

newman run col1.json this contains failure test case
newman run col2.json this passes

when integrated with jenkins as last executed is col2 and it successfull the build result is success, but it should be failure, do have any idea

FYI, tried with --bail option but it is not working as expected.

[kunagpal]

@Sripathi1983 --bail will merely stop the collection run when the first error occurs. In order to server your use case better, you would want to change the command to use a single statement, like so:

newman run col1.json --bail && newman run col2.json --bail

[ericmacfa]

My (naive) solution doesn't quite fit @Sripathi1983's use case (OP is using the CLI), but I'll post it for reference:

Newman can be used as a Node.js module. If you're using this method, then notice that newman.run takes a callback argument (which is executed once the collection run finishes).

The callback function is passed, among other things, a summary object. The array of assertion failures is located at summary.run.failures.

Using this information, we can implement a naive solution by simply adding the below code to the callback passed to newman.run:

...
if(summary.run.failures.length !== 0){
    process.exit(-1);
}

The non-zero exit code will then be picked up by Jenkins and be interpreted as a build failure.

[shamasis]

Newman already exits with non zero code on failure. Is that not happening?

[Sripathi1983]

Thanks but it is resolved in a different way

[tonitech]

My solution is to separate your build into several steps. The first step is to build your application. The later steps are newman executions. In your case, after building your application, you can set a shell execution for
newman run col1.json
and the other shell execution for
newman run col2.json

[madhavpandey33]

@Sripathi1983 What was your solution? I got into similar problem not sure how to get around this. Any help would be appreciated. Thanks!