Iglistkit: Travis reports success even when tests fail

[DRosadoYew]

New issue checklist

• [x] I have reviewed the README and documentation
• [x] I have searched existing issues and this is not a duplicate

General information

Unsure if this is deliberate behavior.
To have Travis reporting failure when tests fail I believe we would have to add set -e or similar to the build script, to have it return != 0 if the test command fails.

Here you can see a test failing and Travis reporting success - https://travis-ci.org/DRosadoYew/IGListKit/jobs/180866217
And here the same with set -e, reporting failure - https://travis-ci.org/DRosadoYew/IGListKit/jobs/180867672

[rnystrom]

@DRosadoYew wow great find. Luckily our internal tests would block any failures from officially landing, but this will save time when making PRs. cc @jessesquires

[jessesquires]

That's bizarre. I've never seen this behavior.

I looked at other project's .travis.yml and the setup is similar to ours.

@DRosadoYew -- Any travis docs on this? I couldn't find anything...

[DRosadoYew]

@jessesquires, i guess set -o pipefail has the same effect - https://github.com/Alamofire/Alamofire/blob/master/.travis.yml#L32

I think it's about bash not travis - https://www.gnu.org/software/bash/manual/bashref.html#The-Set-Builtin, but not a bash pro whatsoever.

E.g. https://github.com/facebook/AsyncDisplayKit/blob/master/build.sh#L9

[jessesquires]

@DRosadoYew 👍

Yep, confirmed here too:
https://twitter.com/marmelroy/status/805492772736212998

[jessesquires]

Ahh got it. So pipe (| xcpretty -c;) eats the errors. TIL

[Sherlouk]

@jessesquires For what it's worth there's a small bit on xcpretty's repo about this:

https://github.com/supermarin/xcpretty

Important: If you're running xcpretty on a CI like Travis or Jenkins, you may want to exit with same status code as xcodebuild. CI systems usually use status codes to determine if the build has failed.

$ set -o pipefail && xcodebuild [flags] | xcpretty
#
# OR
#
$ xcodebuild [flags] | xcpretty && exit ${PIPESTATUS[0]}

[supermarin]

@jessesquires xcpretty doesn't eat errors, but unix always exits with the status of the last command in the pipeline (e.g same effect if you just piped to cat).

Like @Sherlouk pointed out, you need either pipefail or tell it which cmd status to exit with

[jessesquires]

Thanks @supermarin ! 👍