Cudf: install error[QST]

[iwatobipen]

Dear developer,
Thanks for developing nice tool. I would like to install cudf. But when I tried to install cudf with conda, I got following error.
cations were found to be incompatible with your CUDA driver:

• feature:/linux-64::__cuda==11.0=0
• feature:|@/linux-64::__cuda==11.0=0

Your installed CUDA driver is: 11.0

My cuder driver version is 450 and nvidia-smi shows cuda version is 11.0. But I installed condatoolkit version 10.1.
So I think actual cuda version of my env is cuda10.1.
Are there any way to install cudf without downgrading nvidia-drive version?
Any comments a/o suggestions will be greatly appreciated.
Thanks in advance.

Taka

[kkraus14]

@iwatobipen those messages related to __cuda are a bug in conda and are typically innocuous. Any chance you could share the full output of your conda install/create command to help troubleshoot?

[iwatobipen]

@kkraus14 Thanks for your prompt reply. Here is a full output when I tried to install cudf.

$ conda install -c rapidsai cudf=0.13
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your CUDA driver:

• feature:/linux-64::__cuda==11.0=0
• feature:|@/linux-64::__cuda==11.0=0

Your installed CUDA driver is: 11.0

And list of conda package which has 'cuda' in their name.

$ conda list | grep cuda
cudatoolkit 10.1.243 h6bb024c_0 nvidia
cudatoolkit-dev 10.1.243 h516909a_3 conda-forge
cudnn 7.6.5 cuda10.1_0
openmm 7.4.2 py37_cuda101_rc_1 omnia

Thanks

[kkraus14]

Can you dump the full output of conda list here?

Do you have a .condarc file that specifies other channels already? If so could you post your channels here as well?

[iwatobipen]

Here is the full list of my env and I don't have a .condarc file now.
Thanks

$ conda list

packages in environment at /home/iwatobipen/miniconda3/envs/chemoinfo:

#

Name Version Build Channel

_libgcc_mutex 0.1 main
_py-xgboost-mutex 2.0 cpu_0
absl-py 0.9.0 py37_0
alembic 1.4.2 py_0
amberlite 16.0 pypi_0 pypi
ambertools 17.0 pypi_0 pypi
ambit 0.3 h137fa24_1 psi4
appdirs 1.4.3 py37h28b3542_0
ase 3.19.2 pypi_0 pypi
asn1crypto 1.3.0 py37_1
attrs 19.3.0 py_0
autograd 1.3 py_0 conda-forge
autograd-gamma 0.4.1 py_0 conda-forge
backcall 0.2.0 py_0
bcrypt 3.1.7 py37h7b6447c_1
black 19.10b0 py_0
blas 1.0 mkl
bleach 3.1.5 py_0
blosc 1.19.0 hd408876_0
bokeh 2.1.1 py37_0
boost-cpp 1.68.0 h11c811c_1000 conda-forge
brotlipy 0.7.0 py37h7b6447c_1000
bzip2 1.0.8 h7b6447c_0
ca-certificates 2020.6.24 0
cairo 1.14.12 h8948797_3
catch2 2.11.2 hc9558a2_0 conda-forge
certifi 2020.6.20 py37_0
cffi 1.14.0 py37he30daa8_1
chardet 3.0.4 py37_1003
chemps2 1.8.9 h8c3debe_0 psi4
clang 10.0.1 default_hde54327_0 conda-forge
clang-tools 10.0.1 default_hde54327_0 conda-forge
clangdev 10.0.1 default_hde54327_0 conda-forge
clangxx 10.0.1 default_hde54327_0 conda-forge
click 7.1.2 py_0
cliff 3.3.0 py_0 conda-forge
cloudpickle 1.5.0 py_0
cmaes 0.6.0 pyhbc3b93e_0 conda-forge
cmd2 0.9.22 py37_0 conda-forge
colorama 0.4.3 py_0
colorlog 4.2.1 py37_0
configparser 5.0.0 py_0
cryptography 2.9.2 py37h1ba5d50_0
cudatoolkit 10.1.243 h6bb024c_0 nvidia
cudatoolkit-dev 10.1.243 h516909a_3 conda-forge
cudnn 7.6.5 cuda10.1_0
cupy 7.7.0 py37h0632833_0 conda-forge
curl 7.69.1 hbc83047_0
cycler 0.10.0 py37_0
cython 0.29.21 py37he6710b0_0
cytoolz 0.10.1 py37h7b6447c_0
dask 2.20.0 py_0
dask-core 2.20.0 py_0
databricks-cli 0.9.1 py_0 conda-forge
dbus 1.13.16 hb2f20db_0
decorator 4.4.2 py_0
deepdiff 3.3.0 py37_1 psi4
defusedxml 0.6.0 py_0
dgl-cu101 0.4.3.post2 pypi_0 pypi
dgllife 0.2.4 pypi_0 pypi
distributed 2.20.0 py37_0
dkh 1.2 h173d85e_2 psi4
docker-py 4.2.2 py37_0
docker-pycreds 0.4.0 py_0
entrypoints 0.3 py37_0
expat 2.2.9 he6710b0_2
fastcache 1.1.0 py37h7b6447c_0
fastrlock 0.4 py37he6710b0_0
fftw3f 3.3.4 2 omnia
flake8 3.8.3 py_0
flask 1.1.2 py_0
fontconfig 2.13.0 h9420a91_0
fpsim2 0.2.3 py37_1_g29b1e09 efelix
freetype 2.10.2 he06d7ca_0 conda-forge
fsspec 0.7.4 py_0
future 0.18.2 py37_1
gau2grid 1.3.1 h035aef0_0 psi4
gdma 2.2.6 h0e1e685_6 psi4
gitdb 4.0.5 py_0
gitpython 3.1.3 py_1
glib 2.65.0 h3eb4bd4_0
googledrivedownloader 0.4 pypi_0 pypi
gorilla 0.3.0 py_0 conda-forge
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb31296c_0
gunicorn 20.0.4 py37_0
h5py 2.10.0 pypi_0 pypi
hdf4 4.2.13 h3ca952b_2
hdf5 1.10.2 hba1933b_1
heapdict 1.0.1 py_0
hyperopt 0.2.4 pypi_0 pypi
icu 58.2 he6710b0_3
idna 2.10 py_0
importlib-metadata 1.7.0 py37_0
importlib_metadata 1.7.0 0
intel-openmp 2020.1 217
ipykernel 5.3.4 py37h5ca1d4c_0
ipython 7.16.1 py37h5ca1d4c_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.5.1 py_0
isodate 0.6.0 pypi_0 pypi
isort 5.0.9 py37_0
itsdangerous 1.1.0 py37_0
jedi 0.17.1 py37_0
jinja2 2.11.2 py_0
joblib 0.16.0 py_0
jpeg 9b h024ee3a_2
jsonpickle 1.4.1 py_0
jsonschema 3.2.0 py37_1
jupyter 1.0.0 py_2 conda-forge
jupyter_client 6.1.6 py_0
jupyter_console 6.1.0 py_0
jupyter_core 4.6.3 py37_0
kiwisolver 1.2.0 py37hfd86e86_0
krb5 1.17.1 h173b8e3_0
lcms2 2.11 h396b838_0
ld_impl_linux-64 2.33.1 h53a641e_7
libboost 1.67.0 h46d08c1_4
libclang 10.0.1 default_hde54327_0 conda-forge
libclang-cpp 10.0.1 default_hde54327_0 conda-forge
libclang-cpp10 10.0.1 default_hde54327_0 conda-forge
libcurl 7.69.1 h20c2e04_0
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libiconv 1.15 h516909a_1006 conda-forge
libint 1.2.1 hb4a4fd4_6 psi4
libllvm10 10.0.1 he513fc3_0 conda-forge
libnetcdf 4.4.1.1 hfc65e7b_11 conda-forge
libpng 1.6.37 hed695b0_1 conda-forge
libpq 12.2 h20c2e04_0
libprotobuf 3.12.3 hd408876_0
libsodium 1.0.18 h7b6447c_0
libssh2 1.9.0 h1ba5d50_1
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
libuuid 1.0.3 h1bed415_2
libxc 4.3.4 h7b6447c_0 psi4
libxcb 1.14 h7b6447c_0
libxgboost 1.1.1 he1b5a44_0 conda-forge
libxml2 2.9.10 he19cac6_1
libxslt 1.1.34 hc22bd24_0
lifelines 0.25.0 py_0 conda-forge
lightgbm 2.3.0 py37he6710b0_0
llvm-tools 10.0.1 he513fc3_0 conda-forge
llvmdev 10.0.1 he513fc3_0 conda-forge
llvmlite 0.33.0 pypi_0 pypi
locket 0.2.0 py37_1
lz4-c 1.9.2 he6710b0_1
lzo 2.10 h7b6447c_2
mako 1.1.3 py_0
markupsafe 1.1.1 py37h14c3975_1
matplotlib 3.3.0 1 conda-forge
matplotlib-base 3.3.0 py37hd478181_1 conda-forge
mccabe 0.6.1 py37_1
mesalib 18.3.1 h590aaf7_0 conda-forge
mistune 0.8.4 py37h14c3975_1001
mkl 2020.1 217
mkl-service 2.3.0 py37he904b0f_0
mkl_fft 1.1.0 py37h23d657b_0
mkl_random 1.1.1 py37h0573a6f_0
ml-metrics 0.1.4 pypi_0 pypi
mlflow 1.2.0 py_1 conda-forge
mmpbsa-py 16.0 pypi_0 pypi
mongodb 4.0.3 h597af5e_0
mongoengine 0.20.0 py37hc8dfbb8_2 conda-forge
more-itertools 8.4.0 py_0
msgpack-c 3.2.0 hc5b1762_0 conda-forge
msgpack-python 1.0.0 py37hfd86e86_1
mypy_extensions 0.4.3 py37_0
nbconvert 5.6.1 py37_1
nbformat 5.0.7 py_0
nccl 2.7.8.1 h51cf6c1_0 conda-forge
ncurses 6.2 he6710b0_1
networkx 2.4 py_1
ngboost 0.2.1 pyh9f0ad1d_0 conda-forge
notebook 6.0.3 py37_0
numba 0.50.1 pypi_0 pypi
numexpr 2.7.1 py37h423224d_0
numpy 1.19.1 py37hbc911f0_0
numpy-base 1.19.1 py37hfa32c7d_0
olefile 0.46 py37_0
openforcefield 0.7.1+45.g6426b42a pypi_0 pypi
openforcefields 1.2.0 py37_0 omnia
openmm 7.4.2 py37_cuda101_rc_1 omnia
openssl 1.1.1g h7b6447c_0
openvr 1.0.17 h6bb024c_1 schrodinger
opt-einsum 3.0.0 py_0 conda-forge
optuna 2.0.0 py_0 conda-forge
packaging 20.4 py_0
packmol-memgen 1.0.5rc0 pypi_0 pypi
pandas 1.0.5 py37h0573a6f_0
pandoc 2.10 0
pandocfilters 1.4.2 py37_1
parmed 3.2.0 pypi_0 pypi
parso 0.7.0 py_0
partd 1.1.0 py_0
pathspec 0.7.0 py_0
patsy 0.5.1 py37_0
pbr 5.4.5 py_0
pcmsolver 1.2.1 py37h142c950_0 psi4
pcre 8.44 he6710b0_0
pdb4amber 1.7.dev0 pypi_0 pypi
pexpect 4.8.0 py37_1
pickleshare 0.7.5 py37_1001
pillow 7.2.0 py37hb39fc2d_0
pint 0.10 py_0 psi4
pip 20.1.1 py37_1
pixman 0.40.0 h7b6447c_0
plotly 4.8.2 py_0
pluggy 0.13.1 py37_0
pmw 2.0.1 py37hc8dfbb8_1002 conda-forge
postgresql 12.2 h20c2e04_0
prettytable 0.7.2 py_3 conda-forge
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.5 py_0
prompt_toolkit 3.0.5 0
protobuf 3.12.3 py37he6710b0_0
psi4 1.3.2+ecbda83 py37h31b3128_0 psi4
psutil 5.7.0 py37h7b6447c_0
psycopg2 2.8.5 py37hb09aad4_1 conda-forge
ptyprocess 0.6.0 py37_0
py 1.9.0 py_0
py-boost 1.67.0 py37h04863e7_4
py-cpuinfo 7.0.0 py_0
py-xgboost 1.1.1 py37hc8dfbb8_0 conda-forge
py3dmol 0.8.0 py_0 conda-forge
pychembldb 0.4.1 dev_0
pycodestyle 2.6.0 py_0
pycparser 2.20 py_2
pydantic 1.5.1 py37h7b6447c_0
pyflakes 2.2.0 py_0
pygments 2.6.1 py_0
pymol 2.5.0a0 pypi_0 pypi
pymongo 3.9.0 py37he6710b0_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyperclip 1.8.0 pyh9f0ad1d_0 conda-forge
pyqt 5.9.2 py37h05f1152_2
pyrsistent 0.16.0 py37h7b6447c_0
pyside2 5.9.0a1 py37h4dc837a_0 conda-forge
pysocks 1.7.1 py37_1
pytables 3.4.4 py37ha205bf6_0
pytest 5.4.3 py37_0
python 3.7.7 hcff3b4d_5
python-dateutil 2.8.1 py_0
python-editor 1.0.4 py_0
python_abi 3.7 1_cp37m conda-forge
pytraj 2.0.5 pypi_0 pypi
pytz 2020.1 py_0
pyyaml 5.3.1 py37h7b6447c_1
pyzmq 19.0.1 py37he6710b0_1
qcelemental 0.4.2 py_0 psi4
qcengine 0.8.2 py_0 conda-forge
qcfractal 0.7.2 py_0 conda-forge
qcportal 0.7.2 py_0 conda-forge
qt 5.9.7 h5867ecd_1
qtconsole 4.7.5 py_0
qtpy 1.9.0 py_0
querystring_parser 1.2.4 py_0 conda-forge
razi 0.0.0 pypi_0 pypi
rdflib 5.0.0 pypi_0 pypi
rdkit 2020.03.3.0 py37hc20afe1_1 rdkit
rdkit-postgresql 2020.03.3.0 h8ea0133_0 rdkit
readline 8.0 h7b6447c_0
regex 2020.6.8 py37h7b6447c_0
requests 2.24.0 py_0
resp 0.8.1 pyha93d1a2_0 psi4
retrying 1.3.3 py37_2
sander 16.0 pypi_0 pypi
scikit-learn 0.23.1 py37h423224d_0
scipy 1.5.0 py37h0b6359f_0
seaborn 0.10.1 1 conda-forge
seaborn-base 0.10.1 py_1 conda-forge
send2trash 1.5.0 py37_0
setuptools 49.2.0 py37_0
simint 0.7 h642920c_1 psi4
simplejson 3.17.0 py37h7b6447c_0
sip 4.19.8 py37hf484d3e_0
six 1.15.0 py_0
smmap 3.0.2 py_0
snappy 1.1.8 he6710b0_0
sortedcontainers 2.2.2 py_0
sqlalchemy 1.3.18 py37h8f50634_0 conda-forge
sqlite 3.32.3 h62c20be_0
sqlparse 0.3.1 py_0
statsmodels 0.11.1 py37h7b6447c_0
stevedore 3.2.0 py37hc8dfbb8_0 conda-forge
tabulate 0.8.3 py37_0
tblib 1.6.0 py_0
terminado 0.8.3 py37_0
testpath 0.4.4 py_0
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 hbc83047_0
toml 0.10.1 py_0
toolz 0.10.0 py_0
torch 1.5.0+cu101 pypi_0 pypi
torch-cluster 1.5.6 pypi_0 pypi
torch-geometric 1.6.0 pypi_0 pypi
torch-scatter 2.0.5 pypi_0 pypi
torch-sparse 0.6.6 pypi_0 pypi
torch-spline-conv 1.2.0 pypi_0 pypi
torchvision 0.6.0+cu101 pypi_0 pypi
tornado 6.0.4 py37h7b6447c_1
tqdm 4.48.0 pypi_0 pypi
traitlets 4.3.3 py37_0
typed-ast 1.4.1 py37h7b6447c_0
typing_extensions 3.7.4.2 py_0
urllib3 1.25.9 py_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py37_1
websocket-client 0.57.0 py37_1
werkzeug 1.0.1 py_0
wheel 0.34.2 py37_0
widgetsnbextension 3.5.1 py37_0
xfeat 0.1.0 dev_0
xgboost 1.1.1 py37h3340039_0 conda-forge
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zeromq 4.3.2 he6710b0_2
zict 2.0.0 py_0
zipp 3.1.0 py_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.5 h9ceee32_0

[kkraus14]

Could you try changing your install command to conda install -c rapidsai -c nvidia -c conda-forge -c defaults cudf=0.14?

From the looks of your environment what stands out most to me is you have pandas 1.0.5 while v0.14 of RAPIDS requires pandas 0.25.3 as well as having a bunch of pip packages installed which can play havoc on solving / finding dependencies properly.

If using a new environment is an option I'd strongly suggest taking that route and using: conda create -c rapidsai -c nvidia -c conda-forge -c defaults cudf=0.14

[iwatobipen]

Sorry for my late reply. I tried to your suggested command but it didn't work. On the other side when I created a new environment, I could install cudf without any problems.

[kkraus14]

Got it. Looks like there's conflicts in your current environment and given the number of packages installed in it this isn't unexpected.

Does creating a new environment work for you or do you need it in this existing environment?

[iwatobipen]

Hopefully, I would like to install cudf in the existing environment but I got following error (it was picked up cudf related).
But new env is acceptable if it is difficult to solve the conflict.
Package pyarrow conflicts for:
cudf=0.14 -> pyarrow=0.15.0
qcportal -> pyarrow[version='>=0.13.0']
qcfractal -> qcfractal-core[version='>=0.13.1,<0.13.2.0a0'] -> pyarrow[version='>=0.13.0']

Thanks.

[kkraus14]

Is there any additional conflicts being shown? That looks solvable with pyarrow 0.15.

[iwatobipen]

Hi, I tried to remove some packages and install cudf in my env and found that new version of rdkit 202003.03 and rdkit-postgresql cause the issue. When I uninstalled rdkit-postgresql and downgrade rdkit version to 2018, installation was succeeded.
And I would like to close the issue.
Thank you for taking your time.

[aniruddhakal]

Could you try changing your install command to conda install -c rapidsai -c nvidia -c conda-forge -c defaults cudf=0.14?

From the looks of your environment what stands out most to me is you have pandas 1.0.5 while v0.14 of RAPIDS requires pandas 0.25.3 as well as having a bunch of pip packages installed which can play havoc on solving / finding dependencies properly.

If using a new environment is an option I'd strongly suggest taking that route and using: conda create -c rapidsai -c nvidia -c conda-forge -c defaults cudf=0.14

worked like a charm for me,
Thanks @kkraus14