Conda-build: conda convert unable to convert OS X package to Windows version

[krinsman]

Actual Behavior

R packages built on OS X seem unable to be converted to Windows versions. (The error happens both for --platform win-32 and --platform win-64.)

The error does not happen with the Python package I tested this issue with on (note the Windows versions here), i.e. the issue (if it is one) does seem exclusive to R packages.

Example Error Output

Converting r-superlearner-2.0_23-r343_0.tar.bz2 from osx-64 to win-64
Traceback (most recent call last):
  File "/opt/anaconda/anaconda3/bin/conda-convert", line 11, in <module>
    sys.exit(main())
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/cli/main_convert.py", line 130, in main
    return execute(sys.argv[1:])
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/cli/main_convert.py", line 126, in execute
    api.convert(f, **args.__dict__)
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/api.py", line 303, in convert
    dry_run=dry_run, dependencies=dependencies)
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/convert.py", line 777, in conda_convert
    dependencies, verbose)
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/convert.py", line 669, in convert_from_unix_to_windows
    update_paths_file(temp_dir, target_platform='win')
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/convert.py", line 374, in update_paths_file
    path['_path'] = update_lib_path(path['_path'], 'win')
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/convert.py", line 228, in update_lib_path
    python_version = retrieve_python_version(path)
  File "/opt/anaconda/anaconda3/lib/python3.6/site-packages/conda_build/convert.py", line 107, in retrieve_python_version
    with open(path_file) as index_file:
FileNotFoundError: [Errno 2] No such file or directory: 'lib/R/library/SuperLearner/ChangeLog/info/index.json'

Expected Behavior

At first I thought that the failure might be due to the Fortran dependencies of some of the packages, but the traceback does not seem to suggest this as much as it suggests difficulties in automatically converting relevant path names/variables. (Also not all of the packages depend on Fortran.)

Thus the expected behavior would be that R packages built on OS X can be converted to Windows versions, so that difficult-to-build packages can be shared with more users.

Steps to Reproduce

Using the R package cvauc as an example and trying to convert to a Windows 32 bit version:

conda skeleton cran cvauc
conda build r-cvauc
conda convert <anaconda install directory>/conda-bld/osx-64/r-cvauc-1.1.0-r343h889e2dd_0.tar.bz2 --platform win-32

Using the R package nnls as an example (whose package has C extensions, maybe due to its Fortran dependencies, whence the -f flag) and trying to convert to a Windows 64 bit version:

conda skeleton cran nnls
conda build r-nnls
conda convert <anaconda install directory>/conda-bld/osx-64/r-nnls-1.4-r343h883f7c5_0.tar.bz2 --platform win-64 -f

Output of conda info

This happens in all of the virtual environments I have created and tested, including "fresh/blank" ones and "base/root" (I doubt I'm using the terminology correctly), so I omit the lines specifying the environment:

            shell level : 1
 populated config files : 
          conda version : 4.5.0
    conda-build version : 3.8.0
         python version : 3.6.4.final.0
       base environment : /opt/anaconda/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/osx-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /opt/anaconda/anaconda3/pkgs
       envs directories : /opt/anaconda/anaconda3/envs
               platform : osx-64
             user-agent : conda/4.5.0 requests/2.18.4 CPython/3.6.4 Darwin/15.6.0 OSX/10.11.6
             netrc file : None
           offline mode : False

[mingwandroid]

This is not supported and never will be. It is impossible to convert binaries between two different OSes.

[krinsman]

@mingwandroid Why is converting to Linux supported then?

(I should have added/mentioned in the original post that platform --all works perfectly until it hits Windows.)

I don't understand the (lack of an) issue.

The exact same procedure works for Python packages, so apparently it is possible to convert binaries between two different OSes for Python. This would seem to be a feature parity issue then.

Obviously I don't understand how conda convert works, hence the stupid questions.

[msarahan]

You're confusing "binaries" with "conda packages." Conda packages may contain binaries. Binaries can not be converted to another platform. They must be rebuilt for another platform. You can convert some python packages because they are only really shuffling paths around to match the different platform. If binaries (.so's, .pyd's, etc) are involved, you can't convert them.

There are some R packages that only include R code. In theory, I think those should be convertible, but I don't know what the status is there.

[mingwandroid]

so apparently it is possible to convert binaries between two different OSes for Python

No, you cannot convert real, OS-level binary software between Linux, macOS and Windows.

Pure-python packages (i.e. those that contain no compiled code) can be converted.

For R, there isn't even any need to convert those packages, you simply mark them as:

build:
  noarch: generic

And they will work on all OSes.

[mingwandroid]

You can tell if a package is suitable for noarch: generic by looking at CRAN, for example for superlearner we see:

https://cran.r-project.org/web/packages/SuperLearner/index.html

NeedsCompilation: no

[krinsman]

@mingwandroid @msarahan

Thank you both for your time and help with these questions, and my apologies for being so ignorant/stupid.

[mingwandroid]

Don't call yourself those things, I didn't think that for what it's worth.

[msarahan]

Yeah, don't be hard on yourself. Everyone is somewhere on the learning curve. Just keep going forward and learning more. Try to ask questions about why something doesn't work more than make statements about things being broken. You'll get better responses that way.