Bazel: Features to solve the problems with dll linking on Windows

[vnvo2409]

Description of the problem / feature request:

Solving the problems with dll on Windows. If a package is dependent on 2 different dlls with the same name but different path, Bazel throw an error because it has to copy 2 dlls into the binary folder which causes name clash. The solution is renaming the dll to name-hash.dll and use a .lib file as an interface library and a .DEF file to actually point to the name-hash.dll

What operating system are you running Bazel on?

Windows 10

Have you found anything relevant by searching the web?

https://github.com/bazelbuild/bazel/issues/11482

Any other information, logs, or outputs that you want to share?

With the guidance of @meteorcloudy , I will implement this feature
This is the progress tracker of this issue

• [x] Add a build option to enable this features. --rename_dll
• [x] Change the dll name on Windows to name-hash.dll, where hash will be calculated based on RepositoryName + the package path the DLL belongs to.
• [x] Add the name to .DEF file by using LIBRARY name-hash
• [x] Testing and document

[meteorcloudy]

Thanks for working on this!

Here are some places you probably want to look at:

Where the dynamic library is declared in CcLibrary.java:
https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcLibrary.java;l=623?q=CcLibrary.java&ss=bazel

Where the dynamic library is declared in CcBinary.java (linkshared = 1):
https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcBinary.java;l=350?q=CcBinary.java&ss=bazel%2Fbazel

Where the DEF file is created:
https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CppHelper.java;drc=61d663a503710c8cb0d03d18f2eb776981b49e17;l=915

Note that, Bazel can create DLL from both cc_library and cc_binary. The difference is, the DLL built by cc_binary will contain all transitive dependencies, while the DLL built by cc_library only contains the srcs declared in the target.

[meteorcloudy]

/cc @oquenchil can help make sure this improve will work for cc_shared_library

[vnvo2409]

@meteorcloudy
Thank you for you suggetions.

In addition, I have tried to build bazel-dev on master on windows and get this error a few days ago.

ERROR: D:/users/vnvo/bazel/bazel-dev/src/main/java/com/google/devtools/build/lib/syntax/BUILD:139:1: no such target '@bazel_tools//tools/jdk:jni':
target 'jni' not declared in package 'tools/jdk' defined by D:/users/vnvo/_bazel_vnvo/3jqcaqfr/external/bazel_tools/tools/jdk/BUILD and referenced by '//src/main/java/com/google/devtools/build/lib/syntax:libcpu_profiler.so'

Build with a release branch is fine.
I think this commit cause the problems but I don't know is it the default behavior or not https://github.com/bazelbuild/bazel/commit/6255cb1b918f6af750fef4e306fac44b6d351c25

[meteorcloudy]

@vnvo2409 Can you show me the command line you used to build Bazel?

[meteorcloudy]

And which version of Bazel were you using? It's better to use newer version of Bazel. Old version may not be able to build the latest master branch.

[vnvo2409]

And which version of Bazel were you using? It's better to use newer version of Bazel. Old version may not be able to build the latest master branch.

Sorry, I use the bazel 2.0 to build. I use bazelisk and it fix the problem. Thank you

[vnvo2409]

@meteorcloudy Could you help me ?

• You think it is better to make a PR for each progress or only one PR at the end.
• Does bazel use some kinds of linting ( clang-format, ... ) ? I have not seen anything in the contributing guide

Thank you

[meteorcloudy]

You think it is better to make a PR for each progress or only one PR at the end.

Yes, I think you can implement it in one PR. But I'm not sure we should change CcBinary, because the target is defined explicitly by users, so they probably expect the DLL name to just be .dll on Windows. However, the DLL built by cc_library is implicit when other cc_binary depends on it and specifies linkstatic = 0.

Does bazel use some kinds of linting ( clang-format, ... ) ? I have not seen anything in the contributing guide

The code will be formatted when we import it into Google, but I don't think the tools are available outside.

[vnvo2409]

@meteorcloudy
I find it is easier and better if we keep the name of the cc_library/cc_binary dll and when we copy to the binary directory using symlinkaction, we change the name to name_hash and edit the name in the def file ( or we could write the name_hash to the def file from the beginning since we will have to change it anyway )

Changing the dll name created by cc_library requires many changes in linking as well.

I don't know what we should do if the users provided their own .def. I think best is we check the name of the dll, warning the user if the name does not match name_hash and automatically replace it. The hash is calculated base on the repository name + package directory so it can be calculated manually.

And all these behaviors are off by default so I think we don't have to worry much about breaking changes.

Update: I have finished an intial implementation of this features by keeping the name and changing the name when copy.

[konste]

Please be careful not to break the existing behavior when the binary file produced by cc_binary is named exactly as cc_binary name attribute. It is often used to produce Windows .DLLs and also to produce OSX .dylibs. For some reason cc_library produces .so files on OSX.

[vnvo2409]

@konste @meteorcloudy I have created a draft PR so you can review what I did.
https://github.com/bazelbuild/bazel/pull/11524

Please be careful not to break the existing behavior when the binary file produced by cc_binary is named exactly as cc_binary name attribute. It is often used to produce Windows .DLLs and also to produce OSX .dylibs. For some reason cc_library produces .so files on OSX.

This will be turned on by a flag --rename_dll so if you don't turn it on, it won't break anything

[vnvo2409]

close by https://github.com/bazelbuild/bazel/commit/474c53094133ab1eb8dce16da036f4290290c77e

[vincyyou]

Description of the problem / feature request:

Solving the problems with dll on Windows. If a package is dependent on 2 different dlls with the same name but different path, Bazel throw an error because it has to copy 2 dlls into the binary folder which causes name clash. The solution is renaming the dll to name-hash.dll and use a .lib file as an interface library and a .DEF file to actually point to the name-hash.dll

What operating system are you running Bazel on?

Windows 10

Have you found anything relevant by searching the web?

11482

Any other information, logs, or outputs that you want to share?

With the guidance of @meteorcloudy , I will implement this feature
This is the progress tracker of this issue

• [x] Add a build option to enable this features. --rename_dll
• [x] Change the dll name on Windows to name-hash.dll, where hash will be calculated based on RepositoryName + the package path the DLL belongs to.
• [x] Add the name to .DEF file by using LIBRARY name-hash
• [x] Testing and document