Tarpaulin: Different behaviour with a binary crate if run with and without --test <target>

[Ch00k]

I noticed that if tarpaulin is run against a binary crate, and a test target is specified with --test <target>, in addition to compiling and running the test executable tarpaulin is also running the crate main executable. When there is no --test <target> specified, tarpaulin does not do that.
To illustrate the behavior I made a minimal repo
Cargo.toml contains a [[test]] section, which maps integration target with tests/test.rs.
The latest Github action run has two jobs: without_target and with_target. In these jobs tarpaulin seems to be compiling and running the main binary in different ways.

without_target:

Nov 02 07:34:52.800  INFO cargo_tarpaulin: running /home/runner/work/foo/foo/target/debug/deps/foo-d43680ff337daf47

with_target:

Nov 02 07:35:03.433  INFO cargo_tarpaulin: running /home/runner/work/foo/foo/target/debug/foo

It looks like in case of without_target the /home/runner/work/foo/foo/target/debug/deps/foo-d43680ff337daf47 is considered a test, while in case of with_target /home/runner/work/foo/foo/target/debug/foo is just a regular executable.

In case of with_target the execution of /home/runner/work/foo/foo/target/debug/foo exits with a non-zero exit code, and the entire tarpaulin's run is considered failed because of this.

In my "real" crate I am using the clap crate, which always returns a non-zero exit code when the binary is run without any arguments, so this behavior of tarpaulin is affecting my tests.

[Ch00k]

@xd009642 Is there a workaround I can apply for the time being if I need to run unit and integration tests separately in CI? In my other repo (a library crate) I have two separate CI jobs running cargo tarpaulin --lib and cargo tarpaulin --test integration. That would obviously not work with a binary crate.

[xd009642]

Hmm in your main.rs you could attributes so when ran under tarpaulin you get an empty main and when running normally you get your original main function:

#[cfg(tarpaulin)]
fn main() {}

#[cfg(not(tarpaulin))]
fn main() {
   // your actual main
}

I'll look at this after work, hopefully it's not a big issue

[xd009642]

Right I've tracked it down, and whats more I had it fixed in some uncommitted changes on a PR I was working on that improves some of that part of the code. I'm just adding tests and finishing off that PR then I'll merge it and do a release tonight if there's time, otherwise do one tomorrow probably

[xd009642]

Right the release is happening right now (once CI finishes). It's fixed on your minimal example and I've added a test to make sure it won't regress.

[Ch00k]

I re-ran the CI in my repo, and can confirm the issue is fixed now. Thank you so much for the quick reaction! 🚀