Deeplabcut: Consider providing an environment.yml for installation

[imagejan]

How about providing an environment.yml file with this repository to simplify installation of all required dependencies with conda?

[AlexEMG]

Good point. We will add this soon (but I guess without TensorFlow for GPU).

Please note that we shared a Docker container that comes with a GPU installation of TensorFlow 1.2 https://github.com/AlexEMG/Docker4DeepLabCut

[AlexEMG]

We added conda environment files on the latest push. For installation with CPU support alone
conda env create -f dlcdependencieswTF1.2.yml
will suffice. See installation guide for more details and how to get GPU support going. Thanks for your suggestion!

[emipetrucci]

I'm getting an error when using the command conda env create -f dlcdependencies.yml (or dlcdependencieswTF1.2.yml) and I cannot seem to find any documentation/forum online about why I'm getting this error.

This is what I typed into the Anaconda Prompt:

C:\Users\labuser\Documents\DeepLab\DeepLabCut>conda env create -f dlcdependencies.yml

This is the output:

Using Anaconda Cloud api site https://api.anaconda.org
Error: Can't process without a name
dlcdependencies.yml is not a valid yaml file.
Environment with requierments.txt file needs a name

It is also odd that the statement about "requirements.txt" is misspelled. Any suggestions on how I can get around this error? I have the directory set to the DeepLabCut folder, and I'm using the statement recommended in the installation guide. If you need more information to understand the error, please let me know.

[AlexEMG]

Thanks for pointing this out (I thought these files should work across platforms). I could reproduce your errors and, Rick Warren provided a yaml file for Windows that contains all dependencies. Please check out the updated installation guide for details.

[imagejan]

@AlexEMG if you leave away the prefix: line containing the platform-specific (and user-specific) path, these files should indeed work across platforms.

[AlexEMG]

Removed the prefix.

However, a few packages have slightly different names as we found out... That's why we'll have one installation file for unix/macos and one for Windows!

[emipetrucci]

Thanks so much @AlexEMG I will try this out today!

[eliezyer]

Hi,

I'm facing the same problem trying to install the GPU support version in windows 10. When creating the environment I get the same message:

SpecNotFound: Can't process without a name

@emipetrucci Could create the environment properly?

[AlexEMG]

Are you sure you are in the correct folder? [i.e. the yaml file is wherever you are in the terminal?]

[eliezyer]

That's exactly it. My mistake, I was running in the DeepLabCut and they are in conda-files

[AlexEMG]

Yes, sorry we moved it!

For future reference:

cd deeplabcut/conda-files
conda env create -f dlcDependenciesFORWINDOWSwTF.yaml

See installation guide for more details

[imagejan]

It seems you removed all conda .yml files with the transition to DLC2 in https://github.com/AlexEMG/DeepLabCut/commit/887814e960ef70d193745c9716c1cf9a871abe33.

I'd like to mention that I (together with @wernle) had good experience installing DeepLabCut on Windows using a simple conda environment definition file like this:

# env-fmi-deeplabcut.yml
# 
# DeepLabCut environment for use on FMI computers
# install: conda env create -f env-fmi-deeplabcut.yml
# update:  conda env update -f env-fmi-deeplabcut.yml
name: fmi-deeplabcut
dependencies:
  - python=3.6
  - tensorflow-gpu==1.12
  - cudnn=7
  - wxpython
  - jupyter
  - pip:
    - deeplabcut

With the above, installation should get as easy as:

1. Install latest Nvidia drivers for your graphics card
2. Run:
   conda env create -f env-fmi-deeplabcut.yml

... instead of the steps outlined in the Installation section of your documentation.

A similar second .yml file can be prepared for installation _without_ GPU. All these conda files should work platform-independently due to conda managing the dependencies. I suggest you include these files to ease installation for your users.

[MMathisLab]

But now one only needs to create a blank conda environment and run pip install deeplabcut, which is much simpler... and installing tensorflow is then also easy, and you can pick which TF you want.

[imagejan]

Fair enough.

A few comments:

But now one only needs to create a blank conda environment

At least specifying python=3.6 is still required, if you don't want to end up with python 2.7 in certain systems.

and run pip install deeplabcut

In my experience (on Windows systems only, sorry...), pip install deeplabcut will _not_ install any of the following:

• jupyter
• wxpython
• tensorflow

In order to allow reproducible research, I would prefer defining all dependencies over relying on ones that may or may not be provided by the local system. Maybe that's much less of an issue on Linux, but for Windows users, conda makes life much easier compared to pip, IMHO.

you can pick which TF you want.

Of course, you can leave out tensorflow[-gpu] from the conda file, and you'll have the same flexibility. For unexperienced users, I'd prefer convention over configuration, though, and aim to provide them with a known-good environment definition.

[AlexEMG]

Point taken, we will discuss it today. (this would also allow to include e.g. https://github.com/Anaconda-Platform/nb_conda_kernels)

[MMathisLab]

(That is strange when you install Anaconda you don’t get Jupyter?! The install itself comes within Jupyter, Jupyter Lab, Spyder).

Indeed, the solution for wxPython and TF/CUDA is different for Windows, MacOS, and Linux (as I say in the installation guide) and I answered more questions/emails once we provided the yml files, so that is why in this pre-release of DLC2 I wasn’t willing to document every possible way one can install this. Glad it’s working for you, and thanks for sharing your solution.

[imagejan]

I answered more questions/emails once we provided the yml files, so that is why in this pre-release of DLC2 I wasn’t willing to document every possible way one can install this.

I assume that the (apparent) need to ship different yml files for different platforms, and the resulting questions, arose because the versions defined in the yml files were too specific, pointing to a specified build instead of just the required version.
For example, instead of:

- numpy-base=1.14.5=py35h5c71026_0

just

- numpy-base=1.14

would have been sufficient, and many of the indirect dependencies could even have been left away entirely, because conda would have resolved them automatically when solving the environment during installation.

[AlexEMG]

Fair enough, yet as we found out then there are differences across OS... with regard to some packages...