Mlflow: UserWarning and YAMLLoadWarning messages appear when running example code in mlflow Docker Container

[jimthompson5802]

System information

• Have I written custom code (as opposed to using a stock example script provided in MLflow): examples/sklearn_elasticnet_wine/
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS 10.14.3
• MLflow installed from (source or binary): Docker Image built from mlflow master sha1 c172fcbd
• MLflow version (run mlflow --version): mlflow, version 0.8.3.dev0
• Python version: Python 2.7.15 :: Anaconda, Inc.
• **npm version (if running the dev UI): N/A
• Exact command to reproduce:

cd examples/sklearn_elasticnet_wine
mlflow run . -P alpha=0.5

Describe the problem

Warning messages now appear when parsing yaml files.

/app/mlflow/tracking/context.py:181: UserWarning: Failure attempting to register context provider "unused": 'module' object has no attribute 'PluginRunContextProvider'
  _run_context_provider_registry.register_entrypoints()
/app/mlflow/utils/environment.py:26: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  env = yaml.load(_conda_header)

The mlflow run itself completes successfully.

Source code / logs

(base) root@91486e3414f1:/app/examples/sklearn_elasticnet_wine# mlflow run . -P alpha=0.5
/app/mlflow/tracking/context.py:181: UserWarning: Failure attempting to register context provider "unused": 'module' object has no attribute 'PluginRunContextProvider'
  _run_context_provider_registry.register_entrypoints()
/app/mlflow/utils/environment.py:26: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  env = yaml.load(_conda_header)
2019/03/17 04:35:12 INFO mlflow.projects: === Created directory /tmp/tmpJWHu8P for downloading remote URIs passed to arguments of type 'path' ===
2019/03/17 04:35:12 INFO mlflow.projects: === Running command 'source activate mlflow-b93852916f9be8ee2359db52b5dfab5589743459 && python train.py 0.5 0.1' in run with ID '0eb8582e9b7748fcbcd14f9668db7db7' ===
/opt/conda/envs/mlflow-b93852916f9be8ee2359db52b5dfab5589743459/lib/python3.6/site-packages/mlflow/utils/environment.py:26: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  env = yaml.load(_conda_header)
Elasticnet model (alpha=0.500000, l1_ratio=0.100000):
  RMSE: 0.7947931019036529
  MAE: 0.6189130834228138
  R2: 0.18411668718221819
2019/03/17 04:35:13 INFO mlflow.projects: === Run (ID '0eb8582e9b7748fcbcd14f9668db7db7') succeeded ===

Root Cause Analysis
It appears the PyYAML package was updated on pypi.org a few days ago. When the Docker image is built, the current version (5.1) from pypi.org is pulled.

mlflow --version
/app/mlflow/tracking/context.py:181: UserWarning: Failure attempting to register context provider "unused": 'module' object has no attribute 'PluginRunContextProvider'
  _run_context_provider_registry.register_entrypoints()
/app/mlflow/utils/environment.py:26: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  env = yaml.load(_conda_header)
mlflow, version 0.8.3.dev0
(base) root@91486e3414f1:/app/examples/sklearn_elasticnet_wine# python
Python 2.7.15 |Anaconda, Inc.| (default, Dec 14 2018, 19:04:19)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yaml
>>> yaml.__version__
'5.1'
>>>

This page describes the condition for the warning messages and corrective actions to take.

Prior to the recent update, the version of PyYAML installed during Docker build was 3.13.

(base) root@a059a95f3dbe:/# mlflow --version
mlflow, version 0.8.2
(base) root@a059a95f3dbe:/# python
Python 3.7.2 (default, Dec 29 2018, 06:19:36)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yaml
>>> yaml.__version__
'3.13'
>>>

The short-term fix for this might be to pin PyYAML==3.13 in the test-requirements.txt file and/or setup.py. The long-term fix is to revise invoking the yaml api to parse the conda.yaml files.

I'm willing to help resolve this issue. I just need guidance on the approach I should take.

Updated: 2019-03-17 with explanation for user warning.

[dorukhansergin]

I just came to create an issue about a similar thing when I saw this. I don't use Docker per se, just Conda. Here is the warning I get:

~/miniconda3/envs/mlflow-9c11590e267466ef952e604ddcc6ef166c0a38b9/lib/python3.6/site-packages/mlflow/utils/environment.py:26: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
env = yaml.load(_conda_header)

I went through PyYAMLDocumentation and saw that the recommended way of doing it is using .safe_load instead of .load. Here is the reason why:

Warning: It is not safe to call yaml.load with any data received from an untrusted source! yaml.load is as powerful as pickle.load and so may call any Python function. Check the yaml.safe_load function though.

Since conda.yamls for Mlprojects are simply nested dictionaries of lists of strings, I think it's okay to switch that function to safe_load. Again, this is a decision to be made by the maintainers.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has not been interacted with for 60 days since it was marked as stale! As such, the issue is now closed. If you have this issue or more information, please re-open the issue!