Conda-build: Build fails on Windows with fatal error in launcher

[vinayak-mehta]

I'm trying to build the following recipe in Windows: meta.yaml. The build fails at the test stage on Line 38 (camelot --help) where it tries to display the help section, with a fatal error in launcher: unable to create process. I've pasted the output in the next section.

Actual Behavior

Fatal error in launcher: Unable to create process using '"c:\users\vinayak\anaconda3\conda-bld\camelot-py_1538996016149\_h_env\python.exe" "C:\Users\vinayak\Anaconda3\envs\python3.6\Scripts\camelot.exe" --help'

Expected Behavior

The test should pass after printing the help section.

Steps to Reproduce

Create a directory camelot-py with the meta.yaml and run conda build --python 3.6 camelot-py.

Output of conda info

conda 4.5.11

(python3.6) D:\camelot-conda>conda info

     active environment : python3.6
    active env location : C:\Users\vinayak\Anaconda3\envs\python3.6
            shell level : 2
       user config file : C:\Users\vinayak\.condarc
 populated config files : C:\Users\vinayak\.condarc
          conda version : 4.5.11
    conda-build version : 3.15.1
         python version : 3.7.0.final.0
       base environment : C:\Users\vinayak\Anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/win-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/win-64
                          https://repo.anaconda.com/pkgs/pro/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
                          https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : C:\Users\vinayak\Anaconda3\pkgs
                          C:\Users\vinayak\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\vinayak\Anaconda3\envs
                          C:\Users\vinayak\AppData\Local\conda\conda\envs
                          C:\Users\vinayak\.conda\envs
               platform : win-64
             user-agent : conda/4.5.11 requests/2.19.1 CPython/3.7.0 Windows/10 Windows/10.0.17134
          administrator : True
             netrc file : None
           offline mode : False

[mingwandroid]

I've noticed when using PIP with conda-build it doesn't seem to create entry points on Windows.

Personally I've stopped using PIP in recipes due to this issue and all the trouble around the 1.10 and 1.18 releases. I would rather we had a policy of using the most simple tool that's capable of building a correct package.

When I run into this I just use setutools or distutils instead.

An alternative is to setup your entry_points in the recipe but I personally do not like doing that as it's a maintenance burden and it's far too easy to forget to add new entry_points.

I don't know if anyone can shed some light on this matter, is there some option we need to pas? Maybe @jjhelmus, @nehaljwani, @msarahan, @jjhelmus, @jakirkham, @ocefpaf or @isuruf know more?

[vinayak-mehta]

The strange thing that I can't wrap my head around is that this was working fine for the first 50% of the builds that I did, but started failing after that, even though I didn't do any major changes to the recipe.

[vinayak-mehta]

Personally I've stopped using PIP in recipes due to this issue and all the trouble around the 1.10 and 1.18 releases.

@mingwandroid Is this a known issue? Can you point me to some of the old threads (or give me keywords to search for them) since I wasn't able to find anything. Is it good practice to specify the pip version in the recipe since conda seems to have 1.10 for Python 3.6?

When I run into this I just use setutools or distutils instead.

You mean something like a python setup.py install then? Is there an option to not install the dependencies with that? Will look for it, since conda-forge recommends that all your Python dependencies be installed from conda itself.

An alternative is to setup your entry_points in the recipe but I personally do not like doing that as it's a maintenance burden and it's far too easy to forget to add new entry_points.

Will try this too, I don't see having another entry point for this package, so if this works I should be good to go!

Thanks for the help!

[vinayak-mehta]

PS: The package builds perfectly on Linux.

[msarahan]

If the package has entry points, I highly recommend doing both what Ray says, and putting entry_points stuff in meta.yaml.

For projects that use setuptools, you should have this as your setup line:

python setup.py install --single-version-externally-managed --record=record.txt

Things that have entry points in setup.py but not in meta.yaml have been broken before. I'm surprised that it ever worked. Entry points that setuptools creates often use a central .pth file, which conda can't deal with, as conda only installs/removes files, and cannot change lines within a file. By listing entry points in meta.yaml, you replace those setuptools entry points with ones that don't depend on the .pth file.

[vinayak-mehta]

@msarahan Thanks for the reply! Specifying entry points and changing the setup line that you suggested worked. Though now the build fails for Python 2.7 on Windows. Should I open a separate issue for this? I get the following error message:

failed to get install actions, retrying. exception was: Unsatisfiable dependencies for platform win-64: {'camelot-py==0.2.3=py27_0'}

I built using conda build --python 2.7 camelot-py. Could this be due to some requirement not being available for Python 2.7 under win-64? I tried installing all of them individually and that seemed to work.

[vinayak-mehta]

How can 'camelot-py==0.2.3=py27_0', which is the package itself, be a dependency?

[vinayak-mehta]

For projects that use setuptools, you should have this as your setup line:
python setup.py install --single-version-externally-managed --record=record.txt

Things that have entry points in setup.py but not in meta.yaml have been broken before.

@msarahan Should this be documented somewhere? Since conda recommends "python setup.py install" (what you suggested) but conda-forge recommends using pip. The conda-forge-linter also pointed it out when I changed from pip to python setup.py install.

[mingwandroid]

We'll it's documented in to different places with differences of opinion. The documentation cannot be aligned until consensus is reached.

[vinayak-mehta]

@mingwandroid @msarahan I'm getting the same unsatisfiable dependencies error for Python 2.7 (win-64) in this conda-forge Appveyor build. Any pointers?

[vinayak-mehta]

I'm opening another issue for the Python 2.7 error since the initial problem is solved. @mingwandroid @msarahan Thanks for the help!