Bazel: MSVC detection issue

[davido]

Description of the problem / feature request:

Building bazel with bazel is failing to detect MSVC:

$ ../bazel-0.24.0rc9-windows-x86_64.exe build src:bazel-dev
Loading:
Loading: 0 packages loaded
Analyzing: target //src:bazel-dev (0 packages loaded, 0 targets configured)
INFO: Analysed target //src:bazel-dev (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 32] [-----] BazelWorkspaceStatusAction stable-status.txt
ERROR: C:/users/davido/projects/bazel/src/main/tools/BUILD:59:1: C++ compilation of rule '//src/main/tools:linux-sandbox' failed (Exit 1)

The target you are compiling requires Visual C++ build tools.
Bazel couldn't find a valid Visual C++ build tools installation on your machine.

Visual C++ build tools seems to be installed at C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC
But Bazel can't find the following tools:
    VCVARSALL.BAT

Please check your installation following https://docs.bazel.build/versions/master/windows.html#using

Target //src:bazel-dev failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 7.110s, Critical Path: 0.15s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

See above. Note that BAZEL_VC env variable is set:

$ echo $BAZEL_VC
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC

What operating system are you running Bazel on?

Windows 10.

What's the output of bazel info release?

Build label: 0.24.0rc9

Have you found anything relevant by searching the web?

Different issues, but nothing specific.

Any other information, logs, or outputs that you want to share?

The file that is reported to be missing exists in different directory:

C:/Program\ Files\ (x86)/Microsoft\ Visual\ Studio/2017/Community/VC/Auxiliary/Build/vcvarsall.bat

Also copying this file from this directory in the directory where the file was expected didn't fix the problem.

I can also start from the menu native Visual Studio x64 tool prompt. The command file is located here:

C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2017\Visual Studio Tools\VC\x64 Native Tools Command Prompt for VS 2017

with this content:

%comspec% /k "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat"

@call "%~dp0vcvarsall.bat" x64 %*

and when sourced the compiler version is reported:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community>cl
Microsoft (R) C/C++-Optimierungscompiler Version 19.16.27027.1 für x64
Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.

Also note, that LibreOffice MSVC detection works as expected on the same VM: http://paste.openstack.org/show/748294.

[meteorcloudy]

@davido Are you sure C:/Program\ Files\ (x86)/Microsoft\ Visual\ Studio/2017/Community/VC/Auxiliary/Build/vcvarsall.bat exists on your machine?
Bazel should be able to find this file for VS 2017.
https://github.com/bazelbuild/bazel/blob/d52f31dab9825dbfe67395eedd5cbef9258be8bb/tools/cpp/windows_cc_configure.bzl#L178-L181

[davido]

@meteorcloudy Last time I checked, the file was there. Question: what is the easiest way to debug the related part of the code? I guess to add some diagnostic "print()" statements?

[meteorcloudy]

Yes, but you'll have to rebuild Bazel after modifying tools/cpp/windows_cc_configure.bzl.
Or you can try setting CC_CONFIGURE_DEBUG=1, see if you can get any useful information.
https://github.com/bazelbuild/bazel/blob/d52f31dab9825dbfe67395eedd5cbef9258be8bb/tools/cpp/windows_cc_configure.bzl#L29-L32

[davido]

Yes, but you'll have to rebuild Bazel after modifying tools/cpp/windows_cc_configure.bzl.

Well, that's the problem here: I cannot patch/build Bazel, because the MSVC detection doesn't work on that VM. I will try with: CC_CONFIGURE_DEBUG. Thanks for the hint.

[davido]

The file is there:

$ file "$BAZEL_VC"/Auxiliary/Build/vcvarsall.bat
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Auxiliary/Build/vcvarsall.bat: ASCII text, with CRLF line terminators

However, I have both MSVC community versions installed on this VM: 14.0 (aka VisuelStudio 2015) and 15.0 (aka as VisualStudio 2017).

If I unset BAZEL_VC environment variable, then bazel can be built sucessfully, but VS 2015 is picked up:

$ ../bazel-0.24.0rc9-windows-x86_64.exe build src:bazel-dev
Loading:
Loading: 0 packages loaded
Analyzing: target //src:bazel-dev (0 packages loaded, 0 targets configured)
DEBUG: C:/users/davido/_bazel_davido/e3lhzu2u/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: 'BAZEL_VC' is not set, start looking for the latest Visual C++ installed.
DEBUG: C:/users/davido/_bazel_davido/e3lhzu2u/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Looking for VS%VERSION%COMNTOOLS environment variables, eg. VS140COMNTOOLS
DEBUG: C:/users/davido/_bazel_davido/e3lhzu2u/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Visual C++ build tools found at C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\
INFO: Analysed target //src:bazel-dev (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Building: checking cached actions
[0 / 1] [-----] BazelWorkspaceStatusAction stable-status.txt
Target //src:bazel-dev up-to-date:
  C:/users/davido/_bazel_davido/e3lhzu2u/execroot/io_bazel/bazel-out/x64_windows-fastbuild/bin/src/bazel-dev
INFO: Elapsed time: 10.134s, Critical Path: 0.01s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action

I suspect there is some ambuguity/mismatch problem with picking the latest MSVS installed on the machine.

If I unset BAZEL_VC then it seems that the detection ov MSVS 2015 is related to VS140COMNTOOLS (that is: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools), however, I have also set VS150COMNTOOLS, that is set to: "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools". As mentioned here: https://stackoverflow.com/questions/43372235/vcvarsall-bat-for-visual-studio-2017, VS2017 has reworked its directory structure and filenames. Particularly, vcvarsqueryregistry.bat does not exist there, but "VsDevCmd.bat" is present.

So I tried to do something like: [1] to detect MSVS 2017, but it didn't work from Bazel. Even though it works as expected from cmd.exe:

C:\Users\davido>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" /no_logo
C:\Users\davido>echo %VCINSTALLDIR%
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\

• [1] http://paste.openstack.org/show/748428/

[davido]

Update:

Now, with BAZEL_VC set to: "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC" it seems to work, and the MSVS 2017 ist detected. I havn't changed anything with the installation, beside experimenting with the patch that I mentioned in previous comment (but it works now with vanila bazel 0.24).

The only issue I have now, is that the output is way to verbose and reports include files resolution: [1].

• [1] http://paste.openstack.org/show/748429/

[rongjiecomputer]

@davido You are seeing all the include file resolution in stdout because Bazel asks MSVC compiler print them out so that Bazel can do header dependency checking.

Bazel will parse output and extract the headers, but Bazel assumes that MSVC compiler is printing in English with each line starting with Note: including file:. From your output, it seems like MSVC compiler is printing in German. You need to configure MSVC to use English locale instead.

https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/ShowIncludesFilter.java;l=57?q=%22%5C%22Note:%22&ss=bazel

[laszlocsomor]

I figured that this string is stored in $VCBINDIR/$LANG/clui.dll as a String Table resource; entry 408.

VS2019 Community Preview comes with 14 language packs. Their prefixes:

Lang ID	String template
1028	注意: 包含檔案:%s%s\n
1029	Poznámka: Včetně souboru:%s%s\n
1031	Hinweis: Einlesen der Datei:%s%s\n
1033	Note: including file:%s%s\n
1036	Remarque : inclusion du fichier : %s%s\n
1040	Nota: file incluso %s%s\n
1041	メモ: インクルード ファイル: %s%s\n
1042	참고: 포함 파일:%s%s\n
1045	Uwaga: w tym pliku: %s%s\n
1046	Observação: incluindo arquivo:%s%s\n
1049	Примечание: включение файла: %s%s\n
1055	Not: eklenen dosya: %s%s\n
2052	注意: 包含文件: %s%s\n
3082	Nota: inclusión del archivo:%s%s\n

[laszlocsomor]

Apparently the only common substring is <: >.

[meteorcloudy]

Maybe we can covert them to utf8 strings and make SHOW_INCLUDES_PREFIX a list?

[laszlocsomor]

SGTM. Do we get the compiler's stdout as UTF-8?

[meteorcloudy]

I think so
https://github.com/bazelbuild/bazel/blob/a1aa5c14782b38d8edf86953230f76ba2662378d/src/main/java/com/google/devtools/build/lib/rules/cpp/ShowIncludesFilter.java#L72

We can test if this works, I will send a change for this.