conda-build doesn't respect --override-channels ?

[beenje]

Actual Behavior

When running conda build with --override-channels --channel conda-forge I would expect conda-build to only look and download packages from conda-forge. We can see below that some packages are downloaded from the defaults channel.

# conda config --show-sources
==> /opt/conda/.condarc <==
channels:
  - conda-forge
  - defaults
show_channel_urls: True

# conda build --override-channels --channel conda-forge recipe
...
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/conda-bld/flask_1567173097489/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho


The following NEW packages will be INSTALLED:

    _libgcc_mutex:   0.1-main             defaults
    bzip2:           1.0.8-h516909a_0     conda-forge
    ca-certificates: 2019.6.16-hecc5488_0 conda-forge
    certifi:         2019.6.16-py37_1     conda-forge
    libffi:          3.2.1-he1b5a44_1006  conda-forge
    libgcc-ng:       9.1.0-hdf63c60_0     defaults
    libstdcxx-ng:    9.1.0-hdf63c60_0     defaults
    ncurses:         6.1-hf484d3e_1002    conda-forge
    openssl:         1.1.1c-h516909a_0    conda-forge
    pip:             19.2.3-py37_0        conda-forge
    python:          3.7.3-h33d41f4_1     conda-forge
    readline:        8.0-hf8c457e_0       conda-forge
    setuptools:      41.2.0-py37_0        conda-forge
    sqlite:          3.29.0-hcee41ef_1    conda-forge
    tk:              8.6.9-hed695b0_1002  conda-forge
    wheel:           0.33.6-py37_0        conda-forge
    xz:              5.2.4-h14c3975_1001  conda-forge
    zlib:            1.2.11-h516909a_1005 conda-forge

Expected Behavior

When using --override-channels conda-build should only use the channels given on the command line.

Steps to Reproduce

Try to build a recipe using --override-channels:

$ conda config --system --set show_channel_urls true
$ git clone https://github.com/conda-forge/flask-feedstock.git
$ cd flask-feedstock
$ conda build --override-channels --channel conda-forge recipe

Output of conda info

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 4.7.11
    conda-build version : 3.18.9
         python version : 3.7.3.final.0
       virtual packages :
       base environment : /opt/conda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.7.11 requests/2.22.0 CPython/3.7.3 Linux/4.9.184-linuxkit debian/10 glibc/2.28
                UID:GID : 0:0
             netrc file : None
           offline mode : False

[marcelotrevisani]

For those specific packages such as libgcc-ng, _libgcc_mutex and libstdcxx-ng conda will get from the main package because conda-forge does not packaged them. Those packages are related to the compiler and conda-forge is using the compilers from pkgs/main

If you try to search for libgcc-ng (with conda-forge channel already added) you will see something like:

➜ conda search libgcc-ng
Loading channels: done
# Name                       Version           Build  Channel             
libgcc-ng                      5.4.0      hdf63c60_3  pkgs/main           
libgcc-ng                      7.2.0      h7cc24e2_2  pkgs/main           
libgcc-ng                      7.2.0      hcbc56d2_1  pkgs/main           
libgcc-ng                      7.2.0      hdf63c60_3  pkgs/main           
libgcc-ng                      7.3.0      hdf63c60_0  pkgs/main           
libgcc-ng                      8.2.0      hdf63c60_0  pkgs/main           
libgcc-ng                      8.2.0      hdf63c60_1  pkgs/main           
libgcc-ng                      9.1.0      hdf63c60_0  pkgs/main  

Did you see the same behavior for another package?

[beenje]

I tried to give a simple example. My use case is with some private channels.
I noticed that the channels in .condarc were used because the "solving environment" should be very fast when using only the given channel. All required packages are available in that channel but some newer packages were still downloaded from defaults. If I remove the extra channels from .condarc, the conda-build command is much faster and all packages are downloaded from the same channel.

In the described example, I'd expect the command to fail (due to some packages not found).
If you run the command with bioconda channel, the result will be the same:
conda build --override-channels --channel bioconda recipe

Nothing will be downloaded from bioconda. Packages will be download from the conda-forge and defaults channels (defined in the .condarc file). The command should fail if the required packages are not available in bioconda.

When using --override-channels, the channels defined in .condarc should be ignored.

[marcelotrevisani]

Indeed, it seems like a bug. I will try to take a look a bit further.
Thanks for reporting it

[mforbes]

Related: how can one ignore .condarc in general? I have tried finding a coherently expressed solution to this problem but don't see it anywhere. I would like to be able to specify the proper channel configurations etc. required to build a package as a config file within that package, not relying on whatever the current user happens to have decided they want to do at the moment with their top-level .condarc file. Is this what --override-channels is for? Can this be included somehow in the local build config files like meta.yaml? The documentation is very confusing on these points.