Conda-build: PREFIX detection on Windows failing

[ryanvolz]

Actual Behavior

Text files with the host prefix are not detected and marked in info/has_prefix.

Expected Behavior

Text files with the host prefix should be detected, marked in info/has_prefix, and the prefix should be replaced upon install.

Steps to Reproduce

Build on Windows with conda-build=3.19.1, see e.g. https://github.com/conda-forge/gobject-introspection-feedstock/issues/19.

Output of conda info

     active environment : None
       user config file : C:\Users\VssAdministrator\.condarc
 populated config files : C:\Users\VssAdministrator\.condarc
          conda version : 4.8.3
    conda-build version : 3.19.1
         python version : 3.6.10.final.0
       virtual packages : 
       base environment : C:\Miniconda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Miniconda\pkgs
                          C:\Users\VssAdministrator\.conda\pkgs
                          C:\Users\VssAdministrator\AppData\Local\conda\conda\pkgs
       envs directories : C:\Miniconda\envs
                          C:\Users\VssAdministrator\.conda\envs
                          C:\Users\VssAdministrator\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.8.3 requests/2.23.0 CPython/3.6.10 Windows/10 Windows/10.0.14393
          administrator : True
             netrc file : None
           offline mode : False

Likely related to #3899. It would appear that the fix in #3903 was not sufficient.

(edited for clarity, "build prefix" -> "host prefix")

[ryanvolz]

I've added a build test at https://github.com/conda-forge/gobject-introspection-feedstock/pull/20 for the failure case that I'm tracking.

In building that gobject-introspection recipe locally, I get proper prefix replacement for conda-build 3.18.11 and 3.19.1 but failure for 3.19.0 (as #3899 would indicate). However, the prefix is still not detected for the conda-forge Azure cloud build even with conda-build=3.19.1, so there is some important difference between the cloud and local build that results in a failure of prefix detection.

[mingwandroid]

We do not replace build prefix on any os.

[mingwandroid]

Please don't refer to the host prefix as the build prefix. They are completely different.

[mingwandroid]

Looking at this now, sorry for the trouble.

[ryanvolz]

Sorry for my sloppy nomenclature, and thanks for taking a look!

[ryanvolz]

https://github.com/conda-forge/yaml-cpp-feedstock/pull/6#issuecomment-608992150 looks to have some more details about how this plays out. I'm not sure if this would still be classed as a conda-build bug, except perhaps that the fallback to regex_files_py fails.

[seanyen]

I think I might discover a regression of the prefix detection logic:

Taking pyqt as an example, when it was built with conda-build 3.18.11, it detected the following files with prefix:

Detected hard-coded path in text file Lib/site-packages/PyQt5-5.12.3.dist-info/RECORD
Detected hard-coded path in text file Lib/site-packages/PyQtWebEngine-5.12.1.dist-info/RECORD
Detected hard-coded path in text file Lib/site-packages/sipconfig.py

And 6 month later, the maintainer added PyChart support and it was built with conda-build 3.19.2. This time, the files detected with the prefix are:

Files containing CONDA_PREFIX
-----------------------------
D:/bld/pyqt_1588403964597/_h_env :: text :: Lib/site-packages/PyQt5-5.12.3.dist-info/RECORD
Lib/site-packages/PyQt5-5.12.3.dist-info/RECORD (text): Patching
D:/bld/pyqt_1588403964597/_h_env :: text :: Lib/site-packages/PyQtChart-5.12.dist-info/RECORD
Lib/site-packages/PyQtChart-5.12.dist-info/RECORD (text): Patching
D:/bld/pyqt_1588403964597/_h_env :: text :: Lib/site-packages/PyQtWebEngine-5.12.1.dist-info/RECORD
Lib/site-packages/PyQtWebEngine-5.12.1.dist-info/RECORD (text): Patching

And you can notice sipconfig.py no longer in the list, but it does contain many hard-coded prefix. Here is an example of the content:

    'default_bin_dir':    'D:\\bld\\pyqt_1588676793556\\_h_env',
    'default_mod_dir':    'D:\\bld\\pyqt_1588676793556\\_h_env\\Lib\\site-packages',
    'default_sip_dir':    'D:\\bld\\pyqt_1588676793556\\_h_env\\sip',

@mingwandroid, do you just happen to have any idea where it might go wrong? Does conda-build check the case of \\ for a path?

[seanyen]

@mingwandroid Friendly ping.

[mingwandroid]

When we make the 3. 19.4 release, hopefully soon, this should be fixed.

[ocefpaf]

When we make the 3. 19.4 release, hopefully soon, this should be fixed.

What is the roadmap for 2.19.4? Is there something we can do to help? This is blocking a lot of conda-forge's updates. See https://github.com/conda-forge/qgis-feedstock/pull/124