Cudf: [BUG]`conda install -c rapidsai cuml` downgrade the cudf 0.13 to 0.61
Describe the bug
I built cudf first in my anaconda env named rapids which is the default version of 0.13, but when I use conda install -c rapidsai cuml the anaconda reports failed and solve this problem by downgrade cudf 0.13 to 0.61 and downgrade cuda 10.2 to 9.2;
Steps/Code to reproduce bug
`## Package Plan ##
environment location: /home/me/anaconda3/envs/rapids
added / updated specs:
- cuml
The following packages will be downloaded:
package | build
---------------------------|-----------------
cuml-0.6.1 | cuda9.2_py36_0 1.8 MB rapidsai
cython-0.29.18 | py36h831f99a_0 2.1 MB conda-forge
libcudf-0.6.1 | cuda9.2_0 13.2 MB rapidsai
libcuml-0.6.1 | cuda9.2_0 19.2 MB rapidsai
pycparser-2.19 | py_2 88 KB conda-forge
------------------------------------------------------------
Total: 36.3 MB
The following NEW packages will be INSTALLED:
cffi conda-forge/linux-64::cffi-1.14.0-py36hd463f26_0
cuml rapidsai/linux-64::cuml-0.6.1-cuda9.2_py36_0
cython conda-forge/linux-64::cython-0.29.18-py36h831f99a_0
libcudf_cffi rapidsai/linux-64::libcudf_cffi-0.6.1-cuda9.2_py36_0
libcuml rapidsai/linux-64::libcuml-0.6.1-cuda9.2_0
pycparser conda-forge/noarch::pycparser-2.19-py_2
The following packages will be REMOVED:
grpc-cpp-1.23.0-h18db393_0
libnvstrings-0.13.0-cuda10.2_0
The following packages will be UPDATED:
ca-certificates anaconda::ca-certificates-2020.1.1-0 --> conda-forge::ca-certificates-2020.4.5.1-hecc5488_0
The following packages will be SUPERSEDED by a higher-priority channel:
openssl anaconda::openssl-1.1.1g-h7b6447c_0 --> conda-forge::openssl-1.1.1g-h516909a_0
The following packages will be DOWNGRADED:
arrow-cpp 0.15.0-py36h090bef1_2 --> 0.12.1-py36h0e61e49_0
boost-cpp 1.70.0-h8e57a91_2 --> 1.68.0-h11c811c_1000
cudf 0.13.0-py36_0 --> 0.6.1-py36_0
icu 64.2-he1b5a44_1 --> 58.2-hf484d3e_1000
libcudf 0.13.0-cuda10.2_0 --> 0.6.1-cuda9.2_0
libprotobuf 3.8.0-h8b12597_0 --> 3.6.1-hdbcaa40_1001
nvstrings 0.13.0-py36_0 --> 0.3.0-cuda9.2_py36_18
pyarrow 0.15.0-py36h8b68381_1 --> 0.12.1-py36hbbcf98d_0
thrift-cpp 0.12.0-hf3afdfd_1004 --> 0.12.0-h0a07b25_1002
Proceed ([y]/n)? n
CondaSystemExit: Exiting.
`
Expected behavior
Please suggest me whether it's possible to fix it in env of anaconda? and why.
Environment overview (please complete the following information)
- Environment location: local Ubuntu 20.04
- Method of cuDF install: conda install -c rapidsai cudf (conda=4.8.3)
Environment details
Please run and paste the output of the cudf/print_env.sh script here, to gather any other relevant environment details
`
CMake
***g++***
/usr/bin/g++
g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
***nvcc***
***Python***
/home/me/anaconda3/envs/rapids/bin/python
Python 3.6.10
packages in environment at /home/me/anaconda3/envs/rapids:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 0_gnu conda-forge
arrow-cpp 0.15.0 py36h090bef1_2 conda-forge
attrs 19.3.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
blas 2.14 openblas conda-forge
bleach 3.1.5 pyh9f0ad1d_0 conda-forge
boost-cpp 1.70.0 h8e57a91_2 conda-forge
brotli 1.0.7 he1b5a44_1001 conda-forge
bzip2 1.0.8 h516909a_2 conda-forge
c-ares 1.15.0 h516909a_1001 conda-forge
ca-certificates 2020.1.1 0 anaconda
certifi 2020.4.5.1 py36_0 anaconda
cudatoolkit 10.2.89 hfd86e86_1
cudf 0.13.0 py36_0 rapidsai
cudnn 7.6.5 cuda10.2_0
cupy 7.4.0 py36h4445f8d_2 conda-forge
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
dlpack 0.2 he1b5a44_1 conda-forge
double-conversion 3.1.5 he1b5a44_2 conda-forge
entrypoints 0.3 py36h9f0ad1d_1001 conda-forge
fastavro 0.23.4 py36h8c4c3a4_0 conda-forge
fastrlock 0.4 py36h831f99a_1001 conda-forge
fsspec 0.6.3 py_0 conda-forge
gflags 2.2.2 he1b5a44_1002 conda-forge
glog 0.4.0 h49b9bf7_3 conda-forge
grpc-cpp 1.23.0 h18db393_0 conda-forge
icu 64.2 he1b5a44_1 conda-forge
importlib-metadata 1.6.0 py36h9f0ad1d_0 conda-forge
importlib_metadata 1.6.0 0 conda-forge
ipykernel 5.2.1 py36h95af2a2_0 conda-forge
ipython 7.14.0 py36h9f0ad1d_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jedi 0.17.0 py36h9f0ad1d_0 conda-forge
jinja2 2.11.2 pyh9f0ad1d_0 conda-forge
joblib 0.14.1 py_0 anaconda
jsonschema 3.2.0 py36h9f0ad1d_1 conda-forge
jupyter_client 6.1.3 py_0 conda-forge
jupyter_core 4.6.3 py36h9f0ad1d_1 conda-forge
ld_impl_linux-64 2.34 h53a641e_0 conda-forge
libblas 3.8.0 14_openblas conda-forge
libcblas 3.8.0 14_openblas conda-forge
libcudf 0.13.0 cuda10.2_0 rapidsai
libevent 2.1.10 h72c5cf5_0 conda-forge
libffi 3.2.1 he1b5a44_1007 conda-forge
libgcc-ng 9.2.0 h24d8f2e_2 conda-forge
libgfortran-ng 7.5.0 hdf63c60_6 conda-forge
libgomp 9.2.0 h24d8f2e_2 conda-forge
liblapack 3.8.0 14_openblas conda-forge
liblapacke 3.8.0 14_openblas conda-forge
libllvm8 8.0.1 hc9558a2_0 conda-forge
libnvstrings 0.13.0 cuda10.2_0 rapidsai
libopenblas 0.3.7 h5ec1e0e_6 conda-forge
libprotobuf 3.8.0 h8b12597_0 conda-forge
librmm 0.13.0 cuda10.2_0 rapidsai
libsodium 1.0.17 h516909a_0 conda-forge
libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge
llvmlite 0.31.0 py36hfa65bc7_1 conda-forge
lz4-c 1.8.3 he1b5a44_1001 conda-forge
markupsafe 1.1.1 py36h8c4c3a4_1 conda-forge
mistune 0.8.4 py36h8c4c3a4_1001 conda-forge
nbconvert 5.6.1 py36h9f0ad1d_1 conda-forge
nbformat 5.0.6 py_0 conda-forge
nccl 2.6.4.1 hc6a2c23_0 conda-forge
ncurses 6.1 hf484d3e_1002 conda-forge
notebook 6.0.3 py36h9f0ad1d_0 conda-forge
numba 0.48.0 py36hb3f55d8_0 conda-forge
numpy 1.18.4 py36h7314795_0 conda-forge
nvstrings 0.13.0 py36_0 rapidsai
openssl 1.1.1g h7b6447c_0 anaconda
packaging 20.4 pyh9f0ad1d_0 conda-forge
pandas 0.25.3 py36hb3f55d8_0 conda-forge
pandoc 2.9.2.1 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parquet-cpp 1.5.1 2 conda-forge
parso 0.7.0 pyh9f0ad1d_0 conda-forge
pexpect 4.8.0 py36h9f0ad1d_1 conda-forge
pickleshare 0.7.5 py36h9f0ad1d_1001 conda-forge
pip 20.1.1 pyh9f0ad1d_0 conda-forge
prometheus_client 0.7.1 py_0 conda-forge
prompt-toolkit 3.0.5 py_0 conda-forge
ptyprocess 0.6.0 py_1001 conda-forge
pyarrow 0.15.0 py36h8b68381_1 conda-forge
pygments 2.6.1 py_0 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pyrsistent 0.16.0 py36h8c4c3a4_0 conda-forge
python 3.6.10 h8356626_1011_cpython conda-forge
python-dateutil 2.8.1 py_0 conda-forge
python_abi 3.6 1_cp36m conda-forge
pytz 2020.1 pyh9f0ad1d_0 conda-forge
pyzmq 19.0.1 py36h9947dbf_0 conda-forge
re2 2020.04.01 he1b5a44_0 conda-forge
readline 8.0 hf8c457e_0 conda-forge
rmm 0.13.0 py36_0 rapidsai
scikit-learn 0.22.1 py36h22eb022_0 anaconda
scipy 1.4.1 py36habc2bb6_0 anaconda
send2trash 1.5.0 py_0 conda-forge
setuptools 46.4.0 py36h9f0ad1d_0 conda-forge
six 1.14.0 py_1 conda-forge
snappy 1.1.8 he1b5a44_1 conda-forge
sqlite 3.30.1 hcee41ef_0 conda-forge
terminado 0.8.3 py36h9f0ad1d_1 conda-forge
testpath 0.4.4 py_0 conda-forge
thrift-cpp 0.12.0 hf3afdfd_1004 conda-forge
tk 8.6.10 hed695b0_0 conda-forge
tornado 6.0.4 py36h8c4c3a4_1 conda-forge
traitlets 4.3.3 py36h9f0ad1d_1 conda-forge
uriparser 0.9.3 he1b5a44_1 conda-forge
wcwidth 0.1.9 pyh9f0ad1d_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
wheel 0.34.2 py_1 conda-forge
xz 5.2.5 h516909a_0 conda-forge
zeromq 4.3.2 he1b5a44_2 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h516909a_1006 conda-forge
zstd 1.4.3 h3b9ef0a_0 conda-forge
`
Additional context
Add any other context about the problem here.
geometrylearner
All 7 comments
@geometrylearner could you try: conda install -c rapidsai -c nvidia -c conda-forge -c defaults cuml?
Do you have channels configured in your .condarc file?
kkraus14
on 20 May 2020
Perfect answer, it works now, thanks a lot.
geometrylearner
on 22 May 2020
Hi Guys, I`m pretty new to Ubuntu and GPU usage. I had a hard time to make Tensorflow work in Ubuntu 20.04 using NVIDIA GTX 1660 ti, meaning I have properly installed cuda 10.1 and Nvidia 440 drivers plus the cuda toolkit.
Following documentation from RAPIDS I noticed this for RAPIDS packages to be installed:
conda create -n rapids-0.16 -c rapidsai -c nvidia -c conda-forge \
-c defaults rapids=0.16 python=3.7 cudatoolkit=10.1
I'm interested in trying out cuml, not sure if I need to install the whole RAPIDS suite in order to do that.
SInce, there is not a lot of tutorials out there, and reading this issue, do you think I can follow @kkraus14 instructions above, meaning only running:
conda install -c rapidsai -c nvidia -c conda-forge -c defaults cuml
would that be enough for me, or do you think should I add something else to make my installation work?
Sorry for the inconvenience if asking here is not appropriate, but this is the most related thing I have found regarding this. Especially because there are no instructions for Ubuntu 20.04 yet.
Thanks in advance,
Alfonso
Rcubes
on 12 Nov 2020
Hi @Rcubes you could use:
conda create -n rapids-0.16 -c rapidsai -c nvidia -c conda-forge \
-c defaults cuml=0.16 python=3.7 cudatoolkit=10.1
rapids in this context is just a conda package we have created that has all of the other RAPIDS libraries as dependencies so they all get installed in one go. You can replace it with cuml or cudf to install that library for example.
kkraus14
on 12 Nov 2020
Hello, @kkraus14. I have been trying to run the commands depicted in RAPIDS Release Selector, which is basically what you wrote here, but the download of cudatoolkit and cudnn, which both come from the "nvidia" package, gets stuck at 0%. Looking into this, I found out that nvidia packages have really slow download speeds, which would explain my problem, because everything else that comes out of that command I am able to download successfully. I tried installing cudatoolkit separately by following the instructions in NVIDIA's website, which worked perfectly (and without any problem in download speeds), but whenever I tried to run rapids command again, it was unable to detect that I already had cudatoolkit installed and so, it tries to download it once again and the download gets stuck. Nonetheless, I only need cudf and cuml, how would I go about installing only those packages and bypassing the installation of cudatoolkit? Even when I remove it from the command, it still appears in the list of "packages to be downloaded" along with cudnn. Thank you in advance for your help.
CamiloMartinezM
on 20 Nov 2020
The conda packages depend on the cudatoolkit conda package as well. You could try moving the nvidia channel below conda-forge and defaults which should allow cudatoolkit to get pulled from conda-forge and cudnn to get pulled from defaults.
There's no way to install cudf or cuml without these conda packages.
kkraus14
on 20 Nov 2020
Thank you. I did as you asked and put the nvidia channel below conda-forge and defaults and when I ran the command and checked where cudatoolkit would be installed from, it got it from the dafaults channel. However, I was not as lucky with cudnn, it still would download it from nvidia, which doesn't work for me. Is there any other workaround to force cudnn to be installed from any other channel except nvidia?
CamiloMartinezM
on 21 Nov 2020
Related issues
razajafri
路
3Comments
henningpeters
路
3Comments
kkraus14
路
3Comments
ericmjl
路
3Comments
galipremsagar
路
3Comments
Most helpful comment
Perfect answer, it works now, thanks a lot.