Pytorch-lightning: Specify mlflow<1.11

[tmcclintock]

🚀 Feature

Specify mlflow<1.11 in the environment.yml file.

Motivation

As I pointed out in this issue in the mlflow repo, the most recent version of that package creates an mlruns directory at the location of any script that imports it. Importing PL also results in this directory being created. This is problematic since it pollutes development environments, and especially test suites.

Pitch

Specify a maximum mlflow version.

Alternatives

Add mlruns to the gitignore.

[github-actions[bot]]

Hi! thanks for your contribution!, great first issue!

[Borda]

I would add it to gitignore or adjust the MLFlowLogger accordingly, mind send a PR?

[tmcclintock]

Well, it isn't the MLFlowLogger that is the issue. It is the package itself. Just doing import mlflow results in the directory being created when on mlflow>=1.11.0.

I'll gladly PR a line in the gitignore for PL, but projects that use PL will still have this issue that is pretty difficult to track down.

[awaelchli]

@tmcclintock If it is a bug, it has to be fixed by the mlflow team, and then we should add mlflow >= X where X is the version it is fixed in. since mlflow is in the extras, you can control which version you want to install. you can downgrade the library yourself as a temporary fix, until it is resolved on their end. Please correct me if I'm wrong.

[tmcclintock]

@awaelchli of course. I offered on the mlflow issue I made to work on the topic, but they haven't gotten back to me yet. I figured that, for now, while this undesirable behavior is the default that PL should avoid it.

[tmcclintock]

Update - I am working with an mlflow dev to fix this. I will issue a PR here that follows @awaelchli's suggestion when it is solved in mlflow.

[tmcclintock]

The mlflow devs fixed this issue on master as seen here. I don't know when this will make it into a version. I'll loop back around here when it does.

[tmcclintock]

This was fixed in mlflow's recent release, so I'm closing. This issue will only exist for specific minor versions of 1.11 (e.g. 1.11.0)

[awaelchli]

Awesome! thanks for taking care of this.

[Borda]

yeah, but probably we need to skip the [articular version mlflow != 1.11 right?

[tmcclintock]

@Borda it turns out the update in mlflow happened without updating the version, so I think taking no action is fine.