Conda-build: Irrelevant warning about NumPy

[maddenp]

Actual Behavior

A build that does not involve NumPy emits info/warning messages about NumPy:

No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11

Expected Behavior

No irrelevant NumPy messages. Well, I _think_ they're irrelevant -- I can't think how this information would be useful in this case.

Steps to Reproduce

In a directory with this meta.yaml:

package:
  name: aws
  version: 0

requirements:
  run:
    - awscli

do conda build -c conda-forge ..

Output of conda info

     active environment : None
            shell level : 0
       user config file : /home/pmadden/.condarc
 populated config files : 
          conda version : 4.5.11
    conda-build version : 3.15.1
         python version : 3.7.0.final.0
       base environment : /home/pmadden/miniconda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/pmadden/miniconda/pkgs
                          /home/pmadden/.conda/pkgs
       envs directories : /home/pmadden/miniconda/envs
                          /home/pmadden/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.19.1 CPython/3.7.0 Linux/4.15.0-34-generic ubuntu/18.04 glibc/2.27
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

[marscher]

I think this goes further.

When the package depends on NumPy indirectly, e.g. via direct dependency, the numpy variant seems to be ignored, when specified via conda_build_config.yaml.

This file is being parsed three times (two times with an absolute path and one time as cwd file) via the parse_config_file function in variants.py, but then the desired result, e.g. pinning NumPy does not work.

contents of conda_build_config.yaml:

numpy:
  - 1.13

But version 1.11 gets installed during test phase. Is this intended? The NumPy version of the depended package is

[msarahan]

@marscher please link to recipes and packages where you see the problem. I think there might just be misunderstanding on how numpy pinning works.

[marscher]

@msarahan thanks for the quick reply.

Here is the recipe without pinning:
https://github.com/markovmodel/pyemma_tutorials/blob/c99bf25625d5a731173c5aa29101f31fb230caa1/devtools/conda-recipe/meta.yaml
As you see it does not directly depend on numpy.

I tried pinning locally with conda_build_config.yml, and ended up in specifying numpy in the host section with the desired spec.
https://github.com/markovmodel/pyemma_tutorials/blob/master/devtools/conda-recipe/meta.yaml

So probably I just do no get the pinning mechanism right?

[msarahan]

Neither of these recipes will involve a numpy pinning from conda_build_config.yaml. Only direct dependencies in the host or the build section undergo the name matching with variants. Run dependencies are not touched. Run dependencies may be affected by run_exports or pin_run_as_build settings, though.

If you have a numpy minimum version for your tutorial, adding the direct dependency with the version constraint is advisable. Counting on implicit behavior to express that constraint is fragile, as you've discovered.

[msarahan]

Regarding this issue, I agree that it isn't pretty. However, the old behavior of setting a numpy value without notifying the user was worse. At the point this is first raised, conda-build does not even know about the recipe's dependencies. It needs to apply selectors before ever parsing meta.yaml.

Clean-up PRs for this would be welcome. Maybe it could be done by adding an extra argument to the ns_cfg function to skip that warning until some later pass, when conda-build has more info about dependencies.

[marscher]

Thanks again for these explanations!
It seems ns_cfg gets called multiple times, so which call actually will contain the right information? I like the idea about the flag.

[msarahan]

Any call that comes as part of finalize_metadata is probably a good bet. Perhaps add the flag with a default value to ns_cfg that disables the warning, then plumb a True value from somewhere in the finalize_metadata call through to ns_cfg.

[caseyclements]

Perhaps rewording the warning will suffice. Something like this?

np = config.variant.get('numpy')
if not np:
    np = defaults['numpy']
    if config.verbose:
        utils.get_logger(__name__).warn("No numpy version specified in conda_build_config.yaml.  "
                                        "If numpy is a requirement in build or host sections, and no version is specified, it will be set to default numpy value of {}.  "
                                        "In run requirements, the version of an unpinned numpy will be the latest that satisfies constraints".format(defaults['numpy']))