Bazel: Use a custom `sitecustomize.py` / wrapper entry point to manipulate `sys.path`

[brandjon]

There are a few issues that relate to the initialization of sys.path.

• #4939: User site packages should not be added (e.g., /home/brandjon/.local/lib/python3.6/site-packages)
• #5899 / #6532: Imported packages should have lower precedence than standard library
• #7091: The main module's directory (which is usually in the source root) should not be added to sys.path

We may be able to address some of this by generating a custom sitecustomize.py file, and ensuring it's given priority by manipulating the PYTHONPATH in the stub script.

Another alternative is to insert another level of indirection before the user's entry point, by creating a wrapper module that runs in the same Python interpreter process as the user code, and that initializes sys.path appropriately. This may affect the sys environment as seen by user code, e.g. sys.argv[0], though maybe the wrapper module could cover some of that up.

Then the question is, if we have an in-process wrapper module anyway, can we get rid of the stub script (which runs in a separate Python process) and thereby solve issues like #544? Or is there some initialization work that has to be done before the real Python process is launched, and that would be tedious to offload into a non-Python launcher script?

[brycearden]

Is there a roadmap for this feature? The import package precedence issues seems to cause a lot of non-trivial issues when using python3 for me. (Even when using the new python-toolchain flow).

Update after 27.0 release:
This issue still exists and can be replicated with the following simple test using the absl-py module and bazel_rules_python at commit:

## Rules python definition to show version that is being used
git_repository(
        name = "io_bazel_rules_python",
        remote = "https://github.com/bazelbuild/rules_python.git",
        commit = "fdbb17a4118a1728d19e638a5291b4c4266ea5b8",
)

## Bazel target
py_test(
    name = "py_version_test",
    size = "small",
    srcs = ["py_version_test.py"],
    python_version = "PY3",
    deps = [
        requirement("absl-py"),
    ],
)

## Test file
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os
import sys
import enum

# print(sys.path)
# print(enum.__file__)
# print(os.environ["PYTHONPATH"])

from absl import app
from absl import logging

def main(argv):
    if sys.version_info[0] < 3:
        print("not using python3")
        # raise Exception("Must be using Python 3")
    logging.info(sys.version_info)

if __name__ == "__main__":
    app.run(main)

[aiuto]

@brandjon Do you think this is still a P2? If so, do you think we should turn it over to ask for help from the community?