Bentoml: Add python version mismatch warning

[parano]

Is your feature request related to a problem? Please describe.

Currently, when the user saves a BentoML bundle, the bentoml.yml file under the bundle directory contains the Python version in which the bundle was created.

When deploying with Docker, bentoml ensures that the python version matches the saved bundle python version. Although when the user is using a custom docker image, or when the user is loading directly with the bentoml.load API, there's no guarantee the python version matches.

This can cause issues in some cases, especially where PickleArtifact or PyTorchArtifact is being used.

We should add a warning when loading a BentoML saved bundle if the current Python version does not match the saved bundle's python version.

Describe the solution you'd like
Add a warning to tell the user about Python version mismatch

Describe alternatives you've considered
Instead of warning, raise an exception.

We can still allow user to specifically set a config to allow loading BentoML bundle of a different python version, e.g. 'BENTOML__CORE__PYPASS_BUNDLE_PY_VERSION_CHECK=true'

Additional context
n/a

[joshuacwnewton]

Hi @parano. I've started to look into this a little. :)

We should add a warning when loading a BentoML saved bundle if the current Python version does not match the saved bundle's python version.

For figuring out the saved bundle's python version, I see there is a python_version in the env category in a saved bundle's bentoml.yml config file. This value comes from the user's installed python version I believe, see the following links:

• [code that saves env config to .yml]
• [code where version is assigned to BentoServiceEnv]
• [code where version is determined from sys.version_info]

But, if we're always taking the python version from sys.version_info, it seems like it wouldn't necessarily be helpful for determining the version of a user-supplied custom base image...

I did some reading (e.g. this page in the documentation), but I am a bit new to Docker, so it is not yet clear to me how to detect the version of python that's inside an arbitrary Docker image.

[parano]

@joshuacwnewton great questions! The problem we are trying to solve is when the training environment and serving environment has a different Python version.

For example, I run the quickstart guide on my laptop which has python 3.6 installed:

cd bentoml
python guides/quick-start/main.py

Now since BentoML takes the python version from current environment, in this saved bundle, the python_version in bentoml.yml file will be 3.6 matching the version of my laptop. The default docker image that BentoML provide, ensures that the docker image will have the identical python version, it essentially installs the target python version with conda: https://github.com/bentoml/BentoML/blob/v0.8.3/bentoml/saved_bundle/bentoml-init.sh

Recently we introduced the custom docker base image API allowing users to use their own docker image for deployment. And if their base image does not have conda installed, BentoML will just use the system python. And it might not be the expected python version.

Let's say I build a custom docker image that has python 3.7 installed and it does not have conda installed, and I specify it to be the base image when creating a BentoML saved bundle on my laptop with Python 3.6. Then when I run the API server docker, sys.version_info in the python session within the docker container will return 3.7, but the bentoml.yml file contains 3.6. And that's when we should show a warning to the user, telling them about the version mismatch.

This warning is also meaningful when the user tries to manually deploy to a server without docker, and they might accidentally install a different python version.

[joshuacwnewton]

Apologies for the delay -- I was finishing up with scikit-learn so I could focus on BentoML full-time for the rest of the MLH program. :)

Thank you so much for your patience, and for taking the time to explain, @parano. I realize now that I didn't understand Docker very well when I asked my question. I have learned more since, and your explanation makes much more sense now, so thank you.

Just to confirm, we are warning just on minor version mismatches (e.g. 3.6 vs. 3.7 as you mention) and not micro version mismatches (e.g. 3.7.6 vs. 3.7.5)?

[parano]

@joshuacwnewton great question! I think we should always show the warning even if it's just a micro version mismatch.

This is because we don't know if all the libraries the user use, or even the user's own code are compatible across different python versions. Our PickleArtifact uses cloudpickle and it is not compatible across different micro versions. So it seems best for us to inform the user about this potential issue.