Bazel: Python tests are flaky on Windows

[meteorcloudy]

http://ci.bazel.io/job/bazel-tests/BAZEL_VERSION=latest,PLATFORM_NAME=windows-x86_64/843/console

Bazel sometimes exist with exitcode = 2 unexpectedly

Traceback (most recent call last):
  File "c:\bazel_ci\temp\Bazel.runfiles_l9ffu5\runfiles\io_bazel\src\test\py\bazel\bazel_clean_test.py", line 30, in testBazelClean
    self.assertEqual(exit_code, 0)
AssertionError: 2 != 0

//cc @laszlocsomor Any idea?

[laszlocsomor]

I added more logging to those tests, will be pushed to GitHub soon.

[laszlocsomor]

Until then, I have no idea what could be going on.

[meteorcloudy]

According to the error messages, Bazel sometimes cannot find VC through registry when running in python tests.
This could be a problem caused by https://github.com/bazelbuild/bazel/issues/2675

Traceback (most recent call last):
  File "c:\bazel_ci\temp\Bazel.runfiles_b7xq91\runfiles\io_bazel\src\test\py\bazel\bazel_clean_test.py", line 33, in testBazelClean
    self.AssertExitCode(exit_code, 0, stderr)
  File "c:\bazel_ci\temp\Bazel.runfiles_b7xq91\runfiles\io_bazel\src\test\py\bazel\test_base.py", line 66, in AssertExitCode
    '(end stderr)------------------------------------------',
AssertionError: Bazel exited with 2 (expected 0), stderr:
(start stderr)----------------------------------------
INFO: $TEST_TMPDIR defined: output root default is 'C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591'.
Extracting Bazel installation...
........................................................................
____Loading package: @bazel_tools//tools/jdk
____Loading package: @local_config_xcode//
____Loading package: @local_jdk//
WARNING: C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: 'BAZEL_PYTHON' is not set, start looking for python in PATH.
.
WARNING: C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: Python found at C:/python_27_amd64/files/python.exe
.
WARNING: C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: 'BAZEL_VC' is not set, start looking for the latest Visual C++ installed.
.
WARNING: C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: Looking for VS%VERSION%COMNTOOLS environment variables,eg. VS140COMNTOOLS
.
WARNING: C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: Looking for Visual C++ through registry
.
ERROR: in target '//external:cc_toolchain': no such package '@local_config_cc//': Traceback (most recent call last):
File "C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/cc_configure.bzl", line 32
configure_windows_toolchain(repository_ctx)
File "C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 320, in configure_windows_toolchain
_find_vc_path(repository_ctx)
File "C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 160, in _find_vc_path
auto_configure_fail("Visual C++ build tools not foun...")
File "C:/bazel_ci/temp/_bazel_system/lc40yure/execroot/io_bazel/_tmp/426fee9b66f3f890e99ba9b867f74591/_bazel_system/y6np-3ym/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 30, in auto_configure_fail
fail("\n%sAuto-Configuration Error:%s...))

Auto-Configuration Error: Visual C++ build tools not found on your machine.

[meteorcloudy]

http://ci.bazel.io/job/bazel-tests/BAZEL_VERSION=HEAD,PLATFORM_NAME=windows-x86_64/847/console
Based on CI output, I can confirm it's caused by #2675
The repository.execute result for querying VC 14.0 is

STDOUT(start)



HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7
    14.0

STDOUT(end)

But should be:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7         
    14.0    REG_SZ    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\  

[laszlocsomor]

I don't know why I closed this bug. I still see this problem on my workstation:

• Windows 10
• Bazel 0.8.1
• VS 2013 (VC 12.0) + VC Build Tools 2015 (VC 14.0) installed
• MSYS at c:/msys64, with junction pointing to it from c:/tools/msys64
• running //src/test/py/bazel:bazel_windows_test

I see that https://github.com/bazelbuild/bazel/commit/6a4247b10f5cf040c1a7176498bef69c75b1b286 removed the BAZEL_VC envvar computation from src/test/py/bazel/test_base.py and I don't understand why. I'll send a patch to restore the logic. /cc @meteorcloudy

[meteorcloudy]

I removed it because after https://github.com/bazelbuild/bazel/issues/2675 is fixed, Bazel should be able to detect VC++ installation on its own. Before that, python_tests were flaky because the result of repository.execute is sometimes incomplete.

[laszlocsomor]

You're right, sorry for the false alarm. My attempts to run the test failed probably because of where my MSYS was installed.