Cargo: Output filename collision with dylib on windows release mode

[khyperia]

This repro case looks really convoluted, my apologies, but, it has a (sort of sane) full use case. Linking https://github.com/rust-lang/cargo/issues/6313 because it's referenced in the output log. I'm not sure if this is a rustc bug or a cargo bug, but it feels like a cargo bug, so I'm filing it here - feel free to tell me to file it on the rust repo.

Here's contents of the whole project for the repro:

./Cargo.toml

[workspace]
members = [
    "thedylib",
    "dylibtest"
]

./thedylib/Cargo.toml

[package]
name = "thedylib"
version = "0.1.0"
authors = ["khyperia <[email protected]>"]
edition = "2018"

[lib]
crate-type = ["dylib"]

[dependencies]

./thedylib/src/lib.rs

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

./dylibtest/Cargo.toml

[package]
name = "dylibtest"
version = "0.1.0"
authors = ["khyperia <[email protected]>"]
edition = "2018"

[build-dependencies]
thedylib = { path = "../thedylib" }

./dylibtest/build.rs

fn main(){}

./dylibtest/src/main.rs

fn main() {
    println!("Hello, world!");
}

---

On windows:

cargo build succeeds without error. (what?? why does debug succeed but release fail??)

cargo build --release fails:

Click to expand large amount of output

warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\deps\thedylib.dll
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\thedylib.dll
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\deps\thedylib.dll.lib
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\thedylib.dll.lib
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\deps\thedylib.dll.exp
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\thedylib.dll.exp
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\deps\thedylib.pdb
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)` has the same output filename as the lib target `thedylib` in package `thedylib v0.1.0 (D:\src\dylibtest\thedylib)`.
Colliding filename is: D:\src\dylibtest\target\release\thedylib.pdb
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
   Compiling thedylib v0.1.0 (D:\src\dylibtest\thedylib)
error: linking with `link.exe` failed: exit code: 1104
  |
  = note: "D:\\Apps\\VS\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "D:\\src\\dylibtest\\target\\release\\deps\\thedylib.thedylib.8ocyyr2v-cgu.0.rcgu.o" "/OUT:D:\\src\\dylibtest\\target\\release\\deps\\thedylib.dll" "/DEF:C:\\Users\\khype\\AppData\\Local\\Temp\\rustcJd4Hni\\lib.def" "D:\\src\\dylibtest\\target\\release\\deps\\thedylib.2jin648boaw7s1nu.rcgu.o" "/OPT:REF,ICF" "/DLL" "/IMPLIB:D:\\src\\dylibtest\\target\\release\\deps\\thedylib.dll.lib" "/DEBUG" "/NATVIS:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "/LIBPATH:D:\\src\\dylibtest\\target\\release\\deps" "/LIBPATH:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/LIBPATH:C:\\Users\\khype\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "std-1feb4ba9912f83e4.dll.lib" "C:\\Users\\khype\\AppData\\Local\\Temp\\rustcJd4Hni\\libcompiler_builtins-0f66c8d6b2ebbbc4.rlib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "msvcrt.lib"
  = note: LINK : fatal error LNK1104: cannot open file 'D:\src\dylibtest\target\release\deps\thedylib.dll'


error: aborting due to previous error

error: could not compile `thedylib`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

I'm not too sure what's going on here, but it's really weird. I believe both debug and release work on linux, but I'm not 100% sure.

Output of cargo version:

cargo 1.47.0 (f3c7e066a 2020-08-28)

[ehuss]

Thanks for the report! Can you say more about why you are using a dylib with a build dependency?

The reason for the collision is that thedylib is being built twice, once as a normal dependency, and once as a build dependency, because build dependencies are built without optimization. In the dev profile, the profile settings are the same so it is only built once, but with --release they get split.

A workaround is to use the --target flag. This causes the output of build dependencies to be stored in a separate directory.

The error message could definitely be better in this case. And ideally it wouldn't happen at all, but there are constraints for naming DLLs on windows that make it difficult. I think something like #6668 would probably be a lead towards fixing this (or maybe making it much worse, I forget!). In general, shared library naming causes a fair bit of trouble, so it is difficult to manage.

[khyperia]

Ah right! I was wondering _why_ it was being built twice, that makes sense.

Hopefully the use case will be more clear tomorrow, we're planning on open-sourcing the project super soon, but yeah. Long story short, the build script references the dylib as a convenience for users building the project. Specifically, the dylib is not required directly by the build script, but rather a tool the build script invokes, which dlopens (or the corresponding platform-specific thing) the dylib. It is _not_ required for a normal dependency - I'm kind of surprised build deps are automatically normal deps, is there a way to disable that? It's a little complex, due to the fact that build.rs merely references a helper crate, and the helper crate is the one that references the dylib (and has all the code for invoking the tool, etc.).

As for _why_ it's a cargo reference, we'd like to make the install process as clean and smooth as possible, so we'd like to avoid requiring people to pre-install the dylib before using anything that happens to (transitively) reference something that needs it in the build script. The first solution was to literally call std::process::Command::new("cargo") inside the build script to build the dylib, but, referencing it as a cargo dep seemed both cleaner and more efficient.

[ehuss]

I'm kind of surprised build deps are automatically normal deps, is there a way to disable that?

When you run cargo build in the root of a virtual workspace, it builds all members as normal packages. You can set default-members to change that behavior.

[khyperia]

Ah thanks! And for context, because it's opensource now as of a few hours ago, the project is rust-gpu. "The dylib" is rustc_codegen_spirv, the rustc backend. "The tool that dlopens the dylib" is rustc (invoked by a nested cargo). "The helper crate that references the dylib" is the spirv-builder crate.

The whole convoluted setup is for users, in their build.rs, to say "hey, I want that crate over there to be built as a spir-v module, which I want to include_bytes!() into the current project".