Galaxy: Conda resolver does not locate R package components during tool execution

[gregvonkuster]

Based on this PR https://github.com/galaxyproject/galaxy/pull/2759, community discussions, etc, I have configured my various local Galaxy development environments to use conda dependency resolution for tools, but I've encountered errors for tools that require R.

I have a simple tool that looks like this:

<?xml version="."?>
<tool id="my_tool" name="My tool" version="1.0.0">
    <description></description>
    <requirements>
         <requirement type="package" version="3.3.1">R</requirement>
    </requirements>
    <command>
        <![CDATA[
            Rscript $__tool_directory__/my_tool.R
            -o "$output"
        ]]>
    </command>
    <inputs>
    </inputs>
    <outputs>
        <data name="output" format="RData"/>
    </outputs>
    <help>
    </help>
    <citations>
        <citation></citation>
    </citations>
</tool>

I have made sure that my conda dependency resolver configuration settings in galaxy.ini are correct:

conda_prefix = /home/greg/work/ts_install/tool_dependency_dir/_conda
conda_exec = /home/greg/work/ts_install/tool_dependency_dir/_conda/bin/conda
conda_debug = True
conda_ensure_channels = r,bioconda,iuc
conda_auto_install = True
conda_verbose_install_check=True
conda_auto_init = True

I have conda set as my only dependency resolver in dependency_resolvers_conf.xml:

<dependency_resolvers>
  <conda />
  <conda versionless="true" />
</dependency_resolvers>

When I install the tool from the Tool Shed, the dependency install process works just fine using conda - conda itself and the tool dependencies are installed in the configured location:

$ pwd
/home/greg/work/ts_install/tool_dependency_dir/_conda
(.venv) [greg@bigsky _conda]$ ll
total 68
drwxrwxr-x. 12 greg greg  4096 Aug 12 13:45 .
drwxrwxr-x.  3 greg greg  4096 Aug 11 11:26 ..
drwxrwxr-x.  2 greg greg 12288 Aug 12 13:29 bin
drwxrwxr-x.  2 greg greg  4096 Aug 12 10:34 conda-meta
drwxrwxr-x. 13 greg greg  4096 Aug 12 09:37 envs
drwxrwxr-x. 10 greg greg  4096 Aug 12 10:34 include
drwxrwxr-x.  9 greg greg  4096 Aug 12 10:34 lib
-rw-rw-r--.  1 greg greg  3699 Feb  4  2016 LICENSE.txt
drwxrwxr-x.  3 greg greg  4096 Aug 12 13:29 man
drwxrwxr-x. 93 greg greg 12288 Aug 12 13:33 pkgs
drwxrwxr-x.  2 greg greg  4096 Aug 12 10:34 sbin
drwxrwxr-x.  5 greg greg  4096 Aug 12 10:34 share
drwxrwxr-x.  3 greg greg  4096 Aug 12 10:34 ssl

When starting up Galaxy, the value of the PATH environment variable is this:

/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin

During tool execution, the job's conda environment is created in the job_working_directory:

$ pwd
/home/greg/work/git_workspace/galaxy/database/job_working_directory/000/45
[greg@bigsky 45]$ ll
total 88
drwxrwxr-x.  5 greg greg 4096 Aug 12 13:33 .
drwxrwxr-x. 25 greg greg 4096 Aug 12 13:33 ..
drwxrwxr-x.  9 greg greg 4096 Aug 12 13:33 conda-env
drwxrwxr-x.  7 greg greg 4096 Aug 12 13:33 conda-metadata-env
-rw-rw-r--.  1 greg greg    4 Aug 12 13:33 galaxy_45.ec
-rwxr-xr-x.  1 greg greg 3649 Aug 12 13:33 galaxy_45.sh
-rw-rw-r--.  1 greg greg   11 Aug 12 13:33 __instrument_core_epoch_end
-rw-rw-r--.  1 greg greg   11 Aug 12 13:33 __instrument_core_epoch_start
-rw-rw-r--.  1 greg greg    2 Aug 12 13:33 __instrument_core_galaxy_slots
-rw-rw-r--.  1 greg greg 2575 Aug 12 13:33 metadata_in_HistoryDatasetAssociation_68_dRVcOB
-rw-rw-r--.  1 greg greg 2112 Aug 12 13:33 metadata_in_HistoryDatasetAssociation_69_3ukqBl
-rw-rw-r--.  1 greg greg   20 Aug 12 13:33 metadata_kwds_HistoryDatasetAssociation_68_6Cjd0r
-rw-rw-r--.  1 greg greg   20 Aug 12 13:33 metadata_kwds_HistoryDatasetAssociation_69_mrEBUC
-rw-rw-r--.  1 greg greg   33 Aug 12 13:33 metadata_out_HistoryDatasetAssociation_68_aquoqa
-rw-rw-r--.  1 greg greg   33 Aug 12 13:33 metadata_out_HistoryDatasetAssociation_69_xv4C0S
-rw-rw-r--.  1 greg greg    2 Aug 12 13:33 metadata_override_HistoryDatasetAssociation_68_Q5m8tw
-rw-rw-r--.  1 greg greg    2 Aug 12 13:33 metadata_override_HistoryDatasetAssociation_69_lSDc1M
-rw-rw-r--.  1 greg greg   44 Aug 12 13:33 metadata_results_HistoryDatasetAssociation_68_Y6EdYv
-rw-rw-r--.  1 greg greg   44 Aug 12 13:33 metadata_results_HistoryDatasetAssociation_69_8zyPW2
-rw-------.  1 greg greg   73 Aug 12 13:33 set_metadata_d6wOqu.py
-rwxr-xr-x.  1 greg greg  678 Aug 12 13:33 tool_script.sh
drwxrwxr-x.  2 greg greg 4096 Aug 12 13:33 working

However, the tool throws an exception because it cannot find Rscript:

/home/greg/work/git_workspace/galaxy/database/job_working_directory/000/45/tool_script.sh: line 9: Rscript: command not found

Now if I prepend the location of the installed Rscript to my PATH...

$ echo $PATH
/home/greg/work/ts_install/tool_dependency_dir/_conda/pkgs/r-base-3.3.1-3/bin:/home/greg/work/ts_install/tool_dependency_dir/_conda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin

...and restart Galaxy and execute the tool, Rscript is found and the tool executes just fine.

So, does using conda dependency resolution require special PATH settings, or am I missing something in my configuration that is causing the conda environment created by the job to not have access to all of the R package components?

[bgruening]

Greg can you try:

<requirement type="package" version="3.3.1">r-base</requirement>

and can you tell me what is inside of /home/greg/work/ts_install/tool_dependency_dir/_conda/envs

[gregvonkuster]

I've got some stuff in there from several test installs, here is the contents before making your change to the tool. The contents at this level did not change when I installed your updates to the tool.

drwxrwxr-x.  7 greg greg 4096 Aug 11 11:29 __package__r-getopt@__unversioned__
drwxrwxr-x.  7 greg greg 4096 Aug 11 11:29 __package__r-getopt@__version__1.20.0
drwxrwxr-x.  9 greg greg 4096 Aug 11 11:28 __package__r-jsonlite@__unversioned__
drwxrwxr-x.  7 greg greg 4096 Aug 11 11:28 __package__r-jsonlite@__version__0.9.19
drwxrwxr-x.  7 greg greg 4096 Aug 11 11:29 __package__r-optparse@__unversioned__
drwxrwxr-x.  7 greg greg 4096 Aug 11 11:29 __package__r-optparse@__version__1.3.0
drwxrwxr-x.  7 greg greg 4096 Aug 12 08:10 __package__r@__unversioned__
drwxrwxr-x.  7 greg greg 4096 Aug 12 09:37 __package__R@__unversioned__
drwxrwxr-x.  9 greg greg 4096 Aug 12 08:13 __package__r@__version__3.3.1
drwxrwxr-x.  9 greg greg 4096 Aug 12 09:37 __package__R@__version__3.3.1
drwxrwxr-x.  8 greg greg 4096 Aug 11 11:29 __package__samtools@__unversioned__

And without the special PATH settings, the tool threw the same Rscript not found exception.

In case it helps, the tool is available here: https://testtoolshed.g2.bx.psu.edu/view/greg/bmsb/303a662c7881
It does not produce valid results currently if Rscript is found and the underlying bmsb.R file is executed, but I'm just trying to get past the issue of locating Rscript at execution time.

[bgruening]

And in the folder __package__r@__version__3.3.1/bin is the RScript executable?

[gregvonkuster]

It is located in the pkgs directory of the various R installs:

(.venv) [greg@bigsky _conda]$ pwd
/home/greg/work/ts_install/tool_dependency_dir/_conda
(.venv) [greg@bigsky _conda]$ ll
total 68
drwxrwxr-x. 12 greg greg  4096 Aug 12 14:47 .
drwxrwxr-x.  3 greg greg  4096 Aug 11 11:26 ..
drwxrwxr-x.  2 greg greg 12288 Aug 12 13:29 bin
drwxrwxr-x.  2 greg greg  4096 Aug 12 10:34 conda-meta
drwxrwxr-x. 15 greg greg  4096 Aug 12 14:47 envs
drwxrwxr-x. 10 greg greg  4096 Aug 12 10:34 include
drwxrwxr-x.  9 greg greg  4096 Aug 12 10:34 lib
-rw-rw-r--.  1 greg greg  3699 Feb  4  2016 LICENSE.txt
drwxrwxr-x.  3 greg greg  4096 Aug 12 13:29 man
drwxrwxr-x. 93 greg greg 12288 Aug 12 14:47 pkgs
drwxrwxr-x.  2 greg greg  4096 Aug 12 10:34 sbin
drwxrwxr-x.  5 greg greg  4096 Aug 12 10:34 share
drwxrwxr-x.  3 greg greg  4096 Aug 12 10:34 ssl
(.venv) [greg@bigsky _conda]$ find . | grep Rscript
./pkgs/r-base-3.2.2-0/share/man/man1/Rscript.1
./pkgs/r-base-3.2.2-0/lib/R/bin/Rscript
./pkgs/r-base-3.2.2-0/bin/Rscript
./pkgs/r-base-3.3.1-3/share/man/man1/Rscript.1
./pkgs/r-base-3.3.1-3/lib/R/bin/Rscript
./pkgs/r-base-3.3.1-3/bin/Rscript
./pkgs/r-3.1.2-2/share/man/man1/Rscript.1
./pkgs/r-3.1.2-2/lib/R/bin/Rscript
./pkgs/r-3.1.2-2/bin/Rscript

[mvdbeek]

So based on this output it looks like R ( in capital) does not include Rscript.
If you can change the requirement tag to r-base or r it should (TM) work.

[bgruening]

Greg, the RScript binary needs to be in the envs folder. Please try the r-base package.

[gregvonkuster]

Thanks @bgruening - I removed all R installs from the envs folder and tried a fresh install of the tool using the r-base package.

<tool id="bmsb" name="BMSB" version="1.0.0">
    <description></description>
    <requirements>
         <requirement type="package" version="3.3.1">r-base</requirement>
    </requirements>

This installed the following into envs:

drwxr-xr-x   9 gvk  staff  306 Aug 12 15:53 __package__r-base@__unversioned__
drwxr-xr-x   9 gvk  staff  306 Aug 12 15:53 __package__r-base@__version__3.3.1
drwxr-xr-x   7 gvk  staff  238 Aug 12 09:34 __package__r@__unversioned__
drwxr-xr-x   9 gvk  staff  306 Aug 12 09:33 __package__r@__version__3.3.1
drwxr-xr-x   8 gvk  staff  272 Aug 12 09:34 __package__samtools@__unversioned__

However, Rscript is not installed into the envs folder, but still in the pkgs folder:

$ pwd
/Users/gvk/work/ts_install/tool_dependency_dir/_conda
MacBook-Pro-2:_conda gvk$ find . | grep Rscript
./pkgs/r-3.1.2-1/bin/Rscript
./pkgs/r-3.1.2-1/lib/R/bin/Rscript
./pkgs/r-3.1.2-1/share/man/man1/Rscript.1
./pkgs/r-base-3.3.1-0/bin/Rscript
./pkgs/r-base-3.3.1-0/lib/R/bin/Rscript
./pkgs/r-base-3.3.1-0/share/man/man1/Rscript.1

[bgruening]

It does work for me:

% conda create r-base==3.3.1 -n r331
 % source activate r331
discarding /home/bag/miniconda2/bin from PATH
prepending /home/bag/miniconda2/envs/r331/bin to PATH
 % which Rscript
/home/bag/miniconda2/envs/r331/bin/Rscript

[bgruening]

Oh wait, is this OSX again? Maybe the package is different on OSX? Oo.

[bgruening]

@mvdbeek could you please do the dance two comments above and see if Rscript is in the OSX package?

[gregvonkuster]

@bgruening @mvdbeek Thanks for the help on this! The behavior is the same for me on both OS X and Linux. Perhaps there is something wrong with my configuration?

[gregvonkuster]

The Galaxy job shell script does this:

. /Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin/activate '/Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3018/conda-metadata-env'

This environment does not have access to the Rscript executable:

. /Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin/activate '/Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3018/conda-metadata-env'
discarding /Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin from PATH
prepending /Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3018/conda-metadata-env/bin to PATH
(/Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3018/conda-metadata-env)MacBook-Pro-2:3018 gvk$ Rscript
-bash: Rscript: command not found

[bgruening]

Can you put this tool on github + test data. I will try to run it :)

[mvdbeek]

. /Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin/activate '/Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3018/conda-metadata-env'

This is for the job metadata, activating the actual conda environment happens before this part.
Here is an example

(.venv)mariuss-MBP:~ marius$ source conda/bin/activate r331
discarding /Users/marius/conda/bin from PATH
prepending /Users/marius/conda/envs/r331/bin to PATH
(r331)(.venv)mariuss-MBP:~ marius$ Rscript
Usage: /path/to/Rscript [--options] [-e expr [-e expr2 ...] | file] [args]

--options accepted are
  --help              Print usage and exit

(r331)(.venv)mariuss-MBP:conda marius$ find . -name Rscript
./envs/r331/bin/Rscript
./envs/r331/lib/R/bin/Rscript
./pkgs/r-base-3.3.1-0/bin/Rscript
./pkgs/r-base-3.3.1-0/lib/R/bin/Rscript

[mvdbeek]

Also, can you disable the verbose_install_check option?
IMO this is way too stringent and should probably be removed all-together.
The last thing that comes to mind is a mismatch in conda version.
If you let galaxy manage conda, you should get miniconda in version 3.19.3.

[gregvonkuster]

Thanks for all of the helpful info on this.
@mvdbeek I do have conda installed by the Galaxy / Tool Shed install process, and it is version 3.19.3:

$ pwd
/Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin
$ ./conda --version
conda 3.19.3

There is no other conda on my PATH, so there should be no conflict:

$ conda
-bash: conda: command not found

It looks like you and @bgruening are managing the conda install environment form the command line, not from the Tool Shed install process. Using Bjorn's instructions above from the command line, I can also successfully install R 3.3.1 and Rscript is found:

$ pwd
/Users/gvk/work/ts_install/tool_dependency_dir/_conda

$ ls -l
total 8
drwxr-xr-x   11 gvk  staff   374 Aug 13 08:48 .
drwxr-xr-x   61 gvk  staff  2074 Aug 12 09:32 ..
-rw-r--r--    1 gvk  staff  3699 Mar 25 01:09 LICENSE.txt
drwxr-xr-x   27 gvk  staff   918 Aug 12 09:30 bin
drwxr-xr-x   20 gvk  staff   680 Aug 12 09:30 conda-meta
drwxr-xr-x    7 gvk  staff   238 Aug 12 15:53 envs
drwxr-xr-x   21 gvk  staff   714 Aug 12 09:30 include
drwxr-xr-x   43 gvk  staff  1462 Aug 12 09:30 lib
drwxr-xr-x  106 gvk  staff  3604 Aug 12 15:53 pkgs
drwxr-xr-x    4 gvk  staff   136 Aug 12 09:30 share
drwxr-xr-x    6 gvk  staff   204 Aug 12 09:29 ssl

$ ./bin/conda create r-base==3.3.1 -n r331
Fetching package metadata: ..........
Solving package specifications: .........
Package plan for installation in environment /Users/gvk/work/ts_install/tool_dependency_dir/_conda/envs/r331:

The following NEW packages will be INSTALLED:

    curl:     7.49.0-1
    gsl:      2.1-1   
    icu:      54.1-0  
    jbig:     2.1-0   
    jpeg:     8d-2    
    libgcc:   4.8.5-1 
    libpng:   1.6.22-0
    libtiff:  4.0.6-2 
    libxml2:  2.9.2-1 
    ncurses:  5.9-8   
    openssl:  1.0.2h-1
    pcre:     8.39-0  
    r-base:   3.3.1-0 
    readline: 6.2-2   
    tk:       8.5.18-0
    xz:       5.2.2-0 
    zlib:     1.2.8-3 

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|##################################################################################| 100%
#
# To activate this environment, use:
# $ source activate r331
#
# To deactivate this environment, use:
# $ source deactivate
#

$ source ./bin/activate r331
discarding /Users/gvk/work/ts_install/tool_dependency_dir/_conda/bin from PATH
prepending /Users/gvk/work/ts_install/tool_dependency_dir/_conda/envs/r331/bin to PATH

$ which Rscript
/Users/gvk/work/ts_install/tool_dependency_dir/_conda/envs/r331/bin/Rscript

The problems I've described in this ticket occur when the tool is installed from the Tool Shed and the Galaxy install process controls the installation of the underlying dependencies using conda as the dependency resolver. Specifically, I installed this tool from the TTS: https://testtoolshed.g2.bx.psu.edu/view/greg/bmsb/303a662c7881. I did not use the API for installation, but the Tool Shed UI.

Is this installation process from the Tool Shed something that we're planning to phase out? If not, are you able to successfully install the tool in this way?

Thanks again!! ;)

[mvdbeek]

I can install the tool you mentioned just fine (I deleted all my envs just to be sure)

...
Starting server in PID 51979.
serving on http://127.0.0.1:8080
/Users/marius/src/galaxy/.venv/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:565: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
  'storage.' % (dialect.name, dialect.driver))
127.0.0.1 - - [13/Aug/2016:15:17:43 +0200] "GET /admin_toolshed/browse_repositories HTTP/1.1" 200 - "http://127.0.0.1:8080/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - [13/Aug/2016:15:17:46 +0200] "GET /admin_toolshed/browse_tool_sheds HTTP/1.1" 200 - "http://127.0.0.1:8080/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - [13/Aug/2016:15:17:47 +0200] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=https%3A%2F%2Ftesttoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://127.0.0.1:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - [13/Aug/2016:15:17:59 +0200] "GET /admin_toolshed/prepare_for_install?changeset_revisions=303a662c7881&repository_ids=78ce69219ec3aaf1&tool_shed_url=https%3A%2F%2Ftesttoolshed.g2.bx.psu.edu%2F HTTP/1.1" 200 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
tool_shed.galaxy_install.repository_dependencies.repository_dependency_manager DEBUG 2016-08-13 15:18:05,585 Creating repository dependency objects...
galaxy.tools.toolbox.base DEBUG 2016-08-13 15:18:07,138 Appending to tool panel section: Get Data
tool_shed.util.repository_util DEBUG 2016-08-13 15:18:07,141 Updating an existing row for repository 'bmsb' in the tool_shed_repository table, status set to 'New'.
tool_shed.galaxy_install.repository_dependencies.repository_dependency_manager DEBUG 2016-08-13 15:18:07,146 Building repository dependency relationships...
galaxy.tools.toolbox.base DEBUG 2016-08-13 15:18:07,146 Appending to tool panel section: Get Data
127.0.0.1 - - [13/Aug/2016:15:18:05 +0200] "POST /admin_toolshed/prepare_for_install HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install?changeset_revisions=303a662c7881&repository_ids=78ce69219ec3aaf1&tool_shed_url=https%3A%2F%2Ftesttoolshed.g2.bx.psu.edu%2F" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
galaxy.tools.toolbox.base DEBUG 2016-08-13 15:18:07,378 Appending to tool panel section: Get Data
127.0.0.1 - - [13/Aug/2016:15:18:10 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
galaxy.managers.citations WARNING 2016-08-13 15:18:11,642 Unknown or unspecified citation type: None
galaxy.managers.citations WARNING 2016-08-13 15:18:12,407 Unknown or unspecified citation type: None
galaxy.managers.citations WARNING 2016-08-13 15:18:12,409 Unknown or unspecified citation type: None
galaxy.managers.citations WARNING 2016-08-13 15:18:12,434 Unknown or unspecified citation type: None
galaxy.tools.toolbox.base DEBUG 2016-08-13 15:18:12,461 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/greg/bmsb/bmsb/1.0.0, version: 1.0.0 into tool panel....
galaxy.tools.search DEBUG 2016-08-13 15:18:12,464 Starting to build toolbox index.
galaxy.tools.search DEBUG 2016-08-13 15:18:13,245 Toolbox index finished. It took: 0:00:00.780541
127.0.0.1 - - [13/Aug/2016:15:18:13 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
DEBUG:conda.config:Could not import binstar
DEBUG:conda.fetch:channel_urls=('https://repo.continuum.io/pkgs/free/osx-64/', 'https://repo.continuum.io/pkgs/free/noarch/', 'https://repo.continuum.io/pkgs/pro/osx-64/', 'https://repo.continuum.io/pkgs/pro/noarch/', 'https://conda.anaconda.org/r/osx-64/', 'https://conda.anaconda.org/r/noarch/', 'https://conda.anaconda.org/bioconda/osx-64/', 'https://conda.anaconda.org/bioconda/noarch/', 'https://conda.anaconda.org/bgruening/osx-64/', 'https://conda.anaconda.org/bgruening/noarch/')
Fetching package metadata: INFO:stdoutlog:Fetching package metadata:
DEBUG:requests.packages.urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /bgruening/noarch/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/bgruening/noarch/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /bgruening/osx-64/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/bgruening/osx-64/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /bioconda/noarch/repodata.json.bz2 HTTP/1.1" 200 705
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/bioconda/noarch/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /bioconda/osx-64/repodata.json.bz2 HTTP/1.1" 200 188147
127.0.0.1 - - [13/Aug/2016:15:18:16 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/bioconda/osx-64/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /r/noarch/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/r/noarch/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /r/osx-64/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://conda.anaconda.org/r/osx-64/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): repo.continuum.io
DEBUG:requests.packages.urllib3.connectionpool:"GET /pkgs/pro/noarch/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://repo.continuum.io/pkgs/pro/noarch/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /pkgs/pro/osx-64/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://repo.continuum.io/pkgs/pro/osx-64/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /pkgs/free/noarch/repodata.json.bz2 HTTP/1.1" 304 0
.DEBUG:dotupdate:fetching repodata: args ('https://repo.continuum.io/pkgs/free/noarch/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}
DEBUG:requests.packages.urllib3.connectionpool:"GET /pkgs/free/osx-64/repodata.json.bz2 HTTP/1.1" 304 0
127.0.0.1 - - [13/Aug/2016:15:18:19 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
.DEBUG:dotupdate:fetching repodata: args ('https://repo.continuum.io/pkgs/free/osx-64/',) kwargs {'session': <conda.connection.CondaSession object at 0x103756150>, 'use_cache': False}

INFO:stdoutlog:

DEBUG:conda.plan:Pinned specs=[]
DEBUG:conda.plan:H0 specs=['r-base 3.3.1*']
DEBUG:conda.plan:H2 python False
DEBUG:conda.plan:H2A python
DEBUG:conda.plan:H2 lua False
DEBUG:conda.plan:H2A lua
DEBUG:conda.plan:HF specs=['r-base 3.3.1*']
DEBUG:conda.resolve:specs=['r-base 3.3.1*']  features=set([])
Solving package specifications: INFO:stdoutlog:Solving package specifications:
DEBUG:conda.resolve:Solving for ['r-base 3.3.1*']
DEBUG:conda.resolve:Features: set([])
DEBUG:conda.resolve:Installed: []
DEBUG:conda.resolve:Beginning the pruning process
.DEBUG:dotupdate:r-base: pruned from 6 -> 1
.DEBUG:dotupdate:xz: pruned from 3 -> 1
.DEBUG:dotupdate:libtiff: pruned from 6 -> 5
.DEBUG:dotupdate:Checking for unsatisfiability
DEBUG:conda.resolve:Using alg BDD
DEBUG:conda.resolve:Bisecting the version constraint
.DEBUG:dotupdate:Building the constraint with rhs: [0, 10]
.DEBUG:dotupdate:Checking for solutions with rhs:  [0, 10]
DEBUG:conda.logic:Evaluated value: 1
.DEBUG:dotupdate:Building the constraint with rhs: [0, 0]
.DEBUG:dotupdate:Checking for solutions with rhs:  [0, 0]
.DEBUG:dotupdate:Finding the minimal solution
DEBUG:conda.logic:min_sat using alg: iterate
DEBUG:conda.logic:Iterate ran 20 times
DEBUG:conda.resolve:Older versions in the solution(s):
DEBUG:conda.resolve:[]

Package plan for installation in environment /Users/marius/conda/envs/__package__r-base@__version__3.3.1:

The following NEW packages will be INSTALLED:

    curl:     7.49.0-1
    gsl:      2.1-1
    icu:      54.1-0
    jbig:     2.1-0
    jpeg:     8d-2
    libgcc:   4.8.5-1
    libpng:   1.6.22-0
    libtiff:  4.0.6-2
    libxml2:  2.9.2-1
    ncurses:  5.9-8
    openssl:  1.0.2h-1
    pcre:     8.39-0
    r-base:   3.3.1-0
    readline: 6.2-2
    tk:       8.5.18-0
    xz:       5.2.2-0
    zlib:     1.2.8-3

DEBUG:conda.instructions: PREFIX('/Users/marius/conda/envs/__package__r-base@__version__3.3.1')
DEBUG:conda.instructions: PRINT('Linking packages ...')
Linking packages ...
INFO:print:Linking packages ...
DEBUG:conda.instructions: PROGRESS('17')
INFO:progress.start:17
DEBUG:conda.instructions: LINK(u'icu-54.1-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'icu', 0)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'icu-54.1-0', linktype=1
DEBUG:conda.instructions: LINK(u'jbig-2.1-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'jbig', 1)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'jbig-2.1-0', linktype=1
DEBUG:conda.instructions: LINK(u'jpeg-8d-2 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'jpeg', 2)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'jpeg-8d-2', linktype=1
DEBUG:conda.instructions: LINK(u'libgcc-4.8.5-1 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'libgcc', 3)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'libgcc-4.8.5-1', linktype=1
DEBUG:conda.instructions: LINK(u'openssl-1.0.2h-1 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'openssl', 4)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'openssl-1.0.2h-1', linktype=1
DEBUG:conda.instructions: LINK(u'readline-6.2-2 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'readline', 5)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'readline-6.2-2', linktype=1
DEBUG:conda.instructions: LINK(u'tk-8.5.18-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'tk', 6)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'tk-8.5.18-0', linktype=1
DEBUG:conda.instructions: LINK(u'xz-5.2.2-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'xz', 7)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'xz-5.2.2-0', linktype=1
DEBUG:conda.instructions: LINK(u'zlib-1.2.8-3 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'zlib', 8)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'zlib-1.2.8-3', linktype=1
DEBUG:conda.instructions: LINK(u'curl-7.49.0-1 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'curl', 9)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'curl-7.49.0-1', linktype=1
DEBUG:conda.instructions: LINK(u'gsl-2.1-1 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'gsl', 10)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'gsl-2.1-1', linktype=1
DEBUG:conda.instructions: LINK(u'libpng-1.6.22-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'libpng', 11)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'libpng-1.6.22-0', linktype=1
DEBUG:conda.instructions: LINK(u'libtiff-4.0.6-2 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'libtiff', 12)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'libtiff-4.0.6-2', linktype=1
DEBUG:conda.instructions: LINK(u'libxml2-2.9.2-1 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'libxml2', 13)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'libxml2-2.9.2-1', linktype=1
DEBUG:conda.instructions: LINK(u'ncurses-5.9-8 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'ncurses', 14)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'ncurses-5.9-8', linktype=1
DEBUG:conda.instructions: LINK(u'pcre-8.39-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'pcre', 15)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'pcre-8.39-0', linktype=1
DEBUG:conda.instructions: LINK(u'r-base-3.3.1-0 /Users/marius/conda/pkgs 1')
INFO:progress.update:(u'r-base', 16)
DEBUG:conda.install:pkgs_dir=u'/Users/marius/conda/pkgs', prefix='/Users/marius/conda/envs/__package__r-base@__version__3.3.1', dist=u'r-base-3.3.1-0', linktype=1
[      COMPLETE      ]|#########################################################################################| 100%
INFO:progress.stop:None
DEBUG:conda.instructions: SYMLINK_CONDA('/Users/marius/conda')
#
# To activate this environment, use:
# $ source activate __package__r-base@__version__3.3.1
#
# To deactivate this environment, use:
# $ source deactivate
#
127.0.0.1 - - [13/Aug/2016:15:18:07 +0200] "POST /admin_toolshed/manage_repositories HTTP/1.1" 302 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - [13/Aug/2016:15:18:21 +0200] "GET /admin_toolshed/monitor_repository_installation?tool_shed_repository_ids=a316732806c9a92c HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
127.0.0.1 - - [13/Aug/2016:15:18:22 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"

And this is what I see in the installed tool manager:

Can you post your startup log?
Current conda has a problem if the path to conda is too long, but in that case you should have a warning in the startup log and you shouldn't be able to click the install through dependency resolver (conda) checkbox.

[gregvonkuster]

@mvdbeek I did see the warning in my startup log for the path to the conda install being too long. I was not aware of this issue, so did not pay close attention to my startup log. I shortened my path to be this...

/Users/gvk/work/ts_install/_conda

...and re-installed the tool, which produced this new error.

Traceback (most recent call last):
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/jobs/runners/__init__.py", line 170, in prepare_job
    job_wrapper.prepare()
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/jobs/__init__.py", line 901, in prepare
    self.dependency_shell_commands = self.tool.build_dependency_shell_commands(job_directory=self.working_directory)
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/tools/__init__.py", line 1291, in build_dependency_shell_commands
    metadata=metadata,
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/tools/deps/__init__.py", line 98, in dependency_shell_commands
    **kwds )
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/tools/deps/__init__.py", line 116, in find_dep
    dependency = resolver.resolve( name, version, type, **kwds )
  File "/Users/gvk/work/git_workspace/galaxy/lib/galaxy/tools/deps/resolvers/conda.py", line 166, in resolve
    raise Exception("Conda dependency seemingly installed but failed to build job environment.")
Exception: Conda dependency seemingly installed but failed to build job environment.

I'll try re-installing the tool again from scratch. Sorry for all of the back-and-forth on this. The long path setting for conda is a particularly nasty bug. Sorry I wasn't aware of it. :(

[mvdbeek]

This is the conda_verbose_install_check setting, which is way too strict for the version number.
If you set this to false it should work now.

[gregvonkuster]

Hmm...I actually have set that to False - here are my settings:

conda_prefix = /Users/gvk/work/ts_install/_conda
conda_exec = /Users/gvk/work/ts_install/_conda/bin/conda
conda_debug = True
conda_ensure_channels = r,bioconda,iuc
conda_auto_install = True
conda_verbose_install_check = False
conda_auto_init = True

But I cannot get past the new error:

Exception: Conda dependency seemingly installed but failed to build job environment.

Even after removing my envs and uninstalling and re-installing the tool.

[gregvonkuster]

I completely remove conda (not just the envs) and re-installed the tool, and I am now getting the original exception when I execute the tool:

/Users/gvk/work/git_workspace/galaxy/database/job_working_directory/003/3023/tool_script.sh: line 9: Rscript: command not found

[gregvonkuster]

I'll try deleting my install database and all previous installs into my Galaxy environment from the Tool Shed to start from scratch. Hopefully that will clear things up.

[mvdbeek]

I'll try deleting my install database and all previous installs into my Galaxy environment from the Tool Shed to start from scratch. Hopefully that will clear things up.

That should not be necessary, there is nothing in the process that is stored in the database... but I don't have a good suggestion either. I'll try your settings ...

[remimarenco]

Just to add my 2 cents, I am also having these issues in my Ubuntu 14 / Galaxy release_16.04.

I have spent a bunch of hours trying to figure out why resolver can't see my dependency (for me ucsc-genepredtobiggenepred) without success.

I gave up by adding the tool_dependency of this tool for TS to install it unfortunately.

I will be happy to help trying to identify the issue. Maybe having a discussion more "interactive" could help you Galaxy_Conda guys experts, solving this?

[mvdbeek]

@gregvonkuster can you unset the conda_exec variable? That one I have never used, and it appears that when it's set, conda is missing activate.

@remimarenco is this on the toolshed somewhere to test?

In general, I would recommend using the latest release 16.07 and not changing the defaults, except for conda_prefix (this should be set to something reasonably short).

[mvdbeek]

@gregvonkuster I have used the settings you have used, and indeed it didn't work straight away.
There was no activate script in /Users/gvk/work/ts_install/_conda/bin the first time I tried.
I have then removed the conda_exec setting, deleted the /Users/gvk/work/ts_install/_conda folder and started galaxy back up, and your tool kind-of ran (missing r-optparse, but Rscript was found).

So I thought, hey, conda_exec is the guilty one. I reactivated conda_exec, deleted /Users/gvk/work/ts_install/_conda and tried again ... this time activate was present.

So I'm at a loss here, maybe there was a temporary problem downgrading conda?

[remimarenco]

Thanks @mvdbeek. Yes, I will move also to 16.07.

The tool is here if you want to try: https://testtoolshed.g2.bx.psu.edu/view/remimarenco/hubarchivecreator/9bd2c5cd4890

Thanks for your time!

[mvdbeek]

@remimarenco this is what I get

Can we assume the stuff that has not been found has a different name in conda?!

[remimarenco]

@mvdbeek As Conda is not working properly on my side. The one that are not resolved are the TS dependencies in tool_dependencies.xml :)

[gregvonkuster]

@mvdbeek I've messed around with this a bit more, but have not encountered the issue you describe where activate is missing in ~/bin. I've always had activate. However, I've discovered that if I remove the contents of /Users/gvk/work/ts_install/_conda and uninstall and re-install the tool, conda and the tool dependencies are no re-installed into /Users/gvk/work/ts_install/_conda.

Obviously there are several different issues going on here with using conda dependency resolution, and its not ideal to attempt to contain them within this single ticket. I'm willing to close this ticket since the original issue was due to a path to the installed conda environment that was longer than 53 characters. Perhaps additional, more specific tickets can be opened by the various parties for the additional issue. let me know if you agree.

Thanks!

[mvdbeek]

Hi @gregvonkuster,

yes, please do. For the 50 character issue, we put that in the google document that will accompany the 16.07 release (https://docs.google.com/document/d/1kM09cQmhF14WVBs_Uprmt93aB8kdiBVRkGUmjqfJpso/edit#), and we will put a warning in the galaxy.ini.sample. We hope the underlying issue will be fixed by the conda team shortly.

[gregvonkuster]

OK, by the way, here is the paster log entry regarding not installing the removed conda environment when re-installing the tool - not sure if this is important...

/bin/sh: /Users/gvk/work/ts_install/_conda/bin/conda: No such file or directory
galaxy.tools.deps.resolvers.conda DEBUG 2016-08-15 08:42:26,199 Removing failed conda install of r-base, version '3.3.1'

[mvdbeek]

Oh yes, that's very helpful.
Can you stop galaxy and delete the folder /Users/gvk/work/ts_install/_conda ?
Galaxy will only install conda if that folder does not exist.
So after the next restart you should see

...
galaxy.tools.toolbox.base DEBUG 2016-08-15 15:05:53,786 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/mvdbeek/docker_scriptrunner/docker_scriptrunner/0.1.6, version: 0.1.6 into tool panel....
galaxy.tools.toolbox.base DEBUG 2016-08-15 15:05:53,786 Loading section: bmsb
galaxy.tools.toolbox.base DEBUG 2016-08-15 15:05:53,787 Loading section: hubarchivecreator
galaxy.tools.toolbox.base DEBUG 2016-08-15 15:05:53,866 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/remimarenco/hubarchivecreator/hubArchiveCreator/2.3.0, version: 2.3.0 into tool panel....
Please run using "bash" or "sh", but not "." or "source"
/Users/marius/src/galaxy/database/tmp/conda_installNNpa9O.bash: line 17: return: can only `return' from a function or sourced script
PREFIX=/Users/marius/conda
installing: _cache-0.0-py27_x0 ...
installing: python-2.7.11-0 ...
installing: conda-env-2.4.5-py27_0 ...
installing: openssl-1.0.2d-0 ...
installing: pycosat-0.6.1-py27_0 ...
installing: pyyaml-3.11-py27_1 ...
installing: readline-6.2-2 ...
installing: requests-2.9.0-py27_0 ...
installing: sqlite-3.8.4.1-1 ...
installing: tk-8.5.18-0 ...
installing: yaml-0.1.6-0 ...
installing: zlib-1.2.8-0 ...
installing: conda-3.19.0-py27_0 ...
installing: pycrypto-2.6.1-py27_0 ...
installing: pip-7.1.2-py27_0 ...
installing: wheel-0.26.0-py27_1 ...
installing: setuptools-18.8.1-py27_0 ...
Python 2.7.11 :: Continuum Analytics, Inc.
creating default environment...
installation finished.
Fetching package metadata: ..........
Solving package specifications: .....................
Package plan for installation in environment /Users/marius/conda:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    openssl-1.0.2h             |                1         3.0 MB
    sqlite-3.13.0              |                0         1.4 MB
    zlib-1.2.8                 |                3          89 KB
    python-2.7.12              |                1         9.5 MB
    conda-env-2.5.2            |           py27_0          26 KB
    pycosat-0.6.1              |           py27_1          89 KB
    pyyaml-3.11                |           py27_4         141 KB
    requests-2.10.0            |           py27_0         611 KB
    setuptools-25.1.6          |           py27_0         474 KB
    wheel-0.29.0               |           py27_0          81 KB
    conda-3.19.3               |           py27_0         178 KB
    pip-8.1.2                  |           py27_0         1.5 MB
    ------------------------------------------------------------
                                           Total:        17.0 MB

The following packages will be UPDATED:

    conda:      3.19.0-py27_0 --> 3.19.3-py27_0
    conda-env:  2.4.5-py27_0  --> 2.5.2-py27_0
    openssl:    1.0.2d-0      --> 1.0.2h-1
    pip:        7.1.2-py27_0  --> 8.1.2-py27_0
    pycosat:    0.6.1-py27_0  --> 0.6.1-py27_1
    python:     2.7.11-0      --> 2.7.12-1
    pyyaml:     3.11-py27_1   --> 3.11-py27_4
    requests:   2.9.0-py27_0  --> 2.10.0-py27_0
    setuptools: 18.8.1-py27_0 --> 25.1.6-py27_0
    sqlite:     3.8.4.1-1     --> 3.13.0-0
    wheel:      0.26.0-py27_1 --> 0.29.0-py27_0
    zlib:       1.2.8-0       --> 1.2.8-3

galaxy.tools.search DEBUG 2016-08-15 15:06:42,364 Starting to build toolbox index.

[gregvonkuster]

Ah, thanks for the pointer. I was investigating the code to try to understand why conda was not being re-installed, so you saved me some time. Perhaps an enhancement to the current code that would install conda not only if the ~/_conda directory does not exist, but also if the directory is empty would be useful.

Thanks again! ;)

[mvdbeek]

Yes, we'll do that, good suggestion.
We should probably also check that conda is installed in the correct revision and functional, and display some hint to the admin if that is not the case.