Cargo: `cargo test --all` ignores different feature sets for different members in a workspace

[alekseysidorov]

Hello, here is a minimal example for this issue.

https://github.com/alekseysidorov/rust_bugs/tree/master/cargo_test_all

Core crate toml:

[package]
name = "core"
version = "0.1.0"
authors = ["Aleksey Sidorov <[email protected]>"]

[dependencies]


[features]
default = []
crazy = []

First dep toml

[package]
name = "dep_first"
version = "0.1.0"
authors = ["Aleksey Sidorov <[email protected]>"]

[dependencies]
core = { path="../core", features=["crazy"] }

Second dep toml with different features

[package]
name = "dep_second"
version = "0.1.0"
authors = ["Aleksey Sidorov <[email protected]>"]

[dependencies]
core = { path="../core"}

And try to run cargo test --all in workspace root.

cargo test --all --verbose 
   Compiling core v0.1.0 (file:///Users/aleksey/develop/rust_bugs/cargo_test_all/core)
     Running `rustc --crate-name core core/src/lib.rs --crate-type lib --emit=dep-info,link -g --cfg 'feature="default"' --cfg 'feature="crazy"' -C metadata=39d9f125bda27c50 -C extra-filename=-39d9f125bda27c50 --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps`
     Running `rustc --crate-name core core/src/lib.rs --emit=dep-info,link -g --test --cfg 'feature="default"' --cfg 'feature="crazy"' -C metadata=e54347fe71e91065 -C extra-filename=-e54347fe71e91065 --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps`
   Compiling dep_second v0.1.0 (file:///Users/aleksey/develop/rust_bugs/cargo_test_all/dep_second)
   Compiling dep_first v0.1.0 (file:///Users/aleksey/develop/rust_bugs/cargo_test_all/dep_first)
     Running `rustc --crate-name dep_second dep_second/src/lib.rs --emit=dep-info,link -g --test -C metadata=da6a2289e18e3cde -C extra-filename=-da6a2289e18e3cde --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps --extern core=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps/libcore-39d9f125bda27c50.rlib`
     Running `rustc --crate-name dep_first dep_first/src/lib.rs --emit=dep-info,link -g --test -C metadata=a681b278103164c3 -C extra-filename=-a681b278103164c3 --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps --extern core=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps/libcore-39d9f125bda27c50.rlib`
     Running `rustc --crate-name dep_first dep_first/src/lib.rs --crate-type lib --emit=dep-info,link -g -C metadata=f059f03ca4bbbcf6 -C extra-filename=-f059f03ca4bbbcf6 --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps --extern core=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps/libcore-39d9f125bda27c50.rlib`
error[E0425]: unresolved function `core::i_am_normal`
 --> dep_second/src/lib.rs:9:9
  |
9 |         core::i_am_normal();
  |         ^^^^^^^^^^^^^^^^^ no resolution found

error: aborting due to previous error

Build failed, waiting for other jobs to finish...
error: Could not compile `dep_second`.

Caused by:
  process didn't exit successfully: `rustc --crate-name dep_second dep_second/src/lib.rs --emit=dep-info,link -g --test -C metadata=da6a2289e18e3cde -C extra-filename=-da6a2289e18e3cde --out-dir /Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps -L dependency=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps --extern core=/Users/aleksey/develop/rust_bugs/cargo_test_all/target/debug/deps/libcore-39d9f125bda27c50.rlib` (exit code: 101)

It seems that the core has not been rebuilt with the feature set for second dependency

[alexcrichton]

Due to the way Cargo's architected right now, this is going to be extremely difficult to fix, if at all. In general though this is where Cargo requires crates to have features be additive for them to be composable.

[retep998]

There's two problems here:

1. Not enough documentation telling the user that features are additive and trying to use them in any other way will only result in pain and suffering. There are many crates out there that don't respect this, including -sys crates that use features to choose how a given native library is compiled.
2. There isn't any system in cargo that allows for non-additive configuration of a dependency, thus causing people to incorrectly use the closest thing they can, which is features.

[Dushistov]

@alexcrichton

Due to the way Cargo's architected right now, this is going to be extremely difficult to fix, if at all. In general though this is where Cargo requires crates to have features be additive for them to be composable.

May be then introduce virtual crates for that?

For example if crateX has Cargo.toml like this:

[breaking-features]
change_everything = []

And if somebody enable change_everything feature for crateX,
then cargo add to dependency tree not "crateX", but "crateX\0change_everything",
and the rest of cargo code works as before?

[nipunn1313]

We ran into this as well, but totally with open source deps
Here is a concrete minimal example

One crate in workspace

[package]
name = "eitherwrap"
version = "0.1.0"
authors = ["Nipunn Koorapati <[email protected]>"]

[dependencies]
either = "1.1"
itertoolswrap = {path = "../itertoolswrap"}

Second crate in workspace

[package]
name = "itertoolswrap"
version = "0.1.0"
authors = ["Nipunn Koorapati <[email protected]>"]

[dependencies]
itertools = "0.6.0"

If you alternate between running cargo test in each of the crates separately, it forces a rebuild each time because of the difference in features (itertools requests no-default-features from either).

nipunn-mbp:featuresworkspace nipunn$ cd eitherwrap/ ; cargo build ; cd ..
   Compiling either v1.1.0
   Compiling itertools v0.6.2
   Compiling itertoolswrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/itertoolswrap)
   Compiling eitherwrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/eitherwrap)
    Finished dev [unoptimized + debuginfo] target(s) in 1.67 secs
nipunn-mbp:featuresworkspace nipunn$ cd itertoolswrap/ ; cargo build ; cd ..
   Compiling either v1.1.0
   Compiling itertools v0.6.2
   Compiling itertoolswrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/itertoolswrap)
    Finished dev [unoptimized + debuginfo] target(s) in 1.49 secs
nipunn-mbp:featuresworkspace nipunn$ cd eitherwrap/ ; cargo build ; cd ..
   Compiling itertools v0.6.2
   Compiling itertoolswrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/itertoolswrap)
   Compiling eitherwrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/eitherwrap)
    Finished dev [unoptimized + debuginfo] target(s) in 1.42 secs
nipunn-mbp:featuresworkspace nipunn$ cd itertoolswrap/ ; cargo build ; cd ..
   Compiling itertools v0.6.2
   Compiling itertoolswrap v0.1.0 (file:///Users/nipunn/src/rust_play/featuresworkspace/itertoolswrap)
    Finished dev [unoptimized + debuginfo] target(s) in 1.30 secs

We have a workspace with 70 crates in it.
The dependency crates we are using do have additive features, but it causes the code to recompile.

Our CI effectively has a loop which effectively runs

for crate in workspace:
    cd crate ; cargo test ; cd -

Since some random subset of our crates ask for either with features and some don't, running tests for all crates forces us to recompile deps repeatedly, which hurts. Especially since itertools is such a low level dep, it's tough. Our CI feels a lot of pain recompiling the same libraries many times.

Looking at our log, for a workspace with 70 crates, we have to recompile itertools 10 different times (which kicks off a chain of recompiles)

Any advice on how to improve the situation?

[nipunn1313]

I think if we included the hashes of all the deps into the metadata for the compiled library, that would improve our problem. At least then the deps would get compiled a max of 2 times in our case, rather than back-n-forth

To illustrate, here are the commands for itertools in the two cases (added newlines for readability and redacted some absolute paths)

Running `rustc --crate-name itertools itertools-0.6.2/src/lib.rs
--crate-type lib --emit=dep-info,link -C debuginfo=2
-C metadata=4ed3e3cf3bc8df3d -C extra-filename=-4ed3e3cf3bc8df3d
--out-dir target/debug/deps
-L dependency=target/debug/deps
--extern either=target/debug/deps/libeither-f93178e8a5af0b1d.rlib
--cap-lints allow`

vs

     Running `rustc --crate-name itertools itertools-0.6.2/src/lib.rs
--crate-type lib --emit=dep-info,link -C debuginfo=2
-C metadata=4ed3e3cf3bc8df3d -C extra-filename=-4ed3e3cf3bc8df3d
--out-dir 
target/debug/deps
-L dependency=
target/debug/deps 
--extern either=target/debug/deps/libeither-4dcab0f19fb09534.rlib
--cap-lints allow`

I don't believe the metadata for itertools should be identical in these two cases because they are clearly linking against different versions of libeither (compiled with different features)

This won't help the cargo test --all issue, but it does seem closely related.

[nipunn1313]

Here is the repro I was working with https://github.com/nipunn1313/cargoissue3620