Tarpaulin: Support conditional compilation under tarpaulin

[jonhoo]

Every now and again, I want to ignore a test when running under tarpaulin, usually because it'll run too slowly with instrumentation. Ideally, this would be as simple as:

#[test]
#[cfg_attr(tarpaulin, ignore)]
fn test_that_tarpaulin_ignores() {}

But unfortunately, #[cfg_attr(tarpaulin, skip)] doesn't generalize beyond specifically that exact invocation.

It'd be nice if tarpaulin always invoked cargo with --cfg=tarpaulin, so that things like the above would work. I _think_ it should just be a matter of adding --cfg=tarpaulin to RUSTFLAGS here
https://github.com/xd009642/tarpaulin/blob/55e1c900bedd42233758633f70cc997b97f0ef7e/src/cargo.rs#L375

[xd009642]

So with --cfg=tarpaulin it seems the idiomatic way to not compile in the test would then be #[cfg(not(tarpaulin))]. I'm just testing this out now :tada:

[xd009642]

So there's now a commit in the develop branch with it in which you can try if you want to feedback at all. Generally how it looks is you'll write the test you don't want to be in the test binary with tarpaulin like:

#[test] 
#[cfg(not(tarpaulin))] 
fn some_big_test() {

}

EDIT: Note to self there's a todo on this, I should make sure that code that might be flagged as potentially missed in any #[cfg(not(tarpaulin))] blocks is ignored in the source_analysis module

[jonhoo]

Yeah, if you don't want the test to even be _compiled_, you'd use

#[cfg(not(tarpaulin))]

If you just want it to not _run_ you'd do

#[cfg_attr(tarpaulin, ignore)]

[xd009642]

Oh duh! Just had that moment of realisation of how cfg_attrs work, I think I was too tunnel visioned into how tarpaulin works I forgot how rust works :joy:

Anyway I'll polish up the source analysis and add a lil bit to the readme about that and then it's all good (I may even push out a release :wink:)

[xd009642]

It's done and 0.13.4 is released with it in. I've also tested it on a few different things so happy that it's all good but feel free to reopen this if there's any issues

[jonhoo]

Does a Docker image also automatically get generated for 0.13.4? Or I guess maybe I should just use latest?

[xd009642]

Use latest I have to add the tag manually to docker hub (I haven't found a way around that in their automated builds unfortunately)

[xd009642]

Just added tags for 0.13.4 so they'll be up whenever docker hub gets to it

[jonhoo]

Huh, interesting, with 0.13.4, this no longer compiles:

#[cfg(test)]
#[cfg_attr(tarpaulin, skip)]
mod tests {
  // ...
}

Presumably because the compiler now turns it into #[skip] (since tarpaulin is a proper cfg now), which in turn (I think) makes tarpaulin's search for cfg_attr(tarpaulin, skip) no longer work. It's also an attribute the compiler doesn't recognize. I _think_ what you want here is probably the attribute:

#[tarpaulin::skip]

But in the meantime, this may have broken some people's code if they relied on skip.

[xd009642]

Oh bugger I'll consider if it's worth yanking 0.13.4 and releasing a new 0.14 tonight because there's a chance that tarpaulin::skip might break my skip filtering. I'll just open this issue to cover the last tidyups

[xd009642]

Just gonna close this in favour of #487