Dvc: Any dvc command throws "TypeError: public() got an unexpected keyword argument 'SEP'"

[Nikesh46]

Bug Report

Please provide information about your setup

Any command on dvc is failing and throwing the following error.
Output of dvc version:
Traceback (most recent call last):
File "/home/dev2/.local/bin/dvc", line 5, in
from dvc.main import main
File "/home/dev2/.local/lib/python3.6/site-packages/dvc/main.py", line 6, in
from dvc import analytics
File "/home/dev2/.local/lib/python3.6/site-packages/dvc/analytics.py", line 16, in
from dvc.lock import Lock, LockError
File "/home/dev2/.local/lib/python3.6/site-packages/dvc/lock.py", line 8, in
import flufl.lock
File "/home/dev2/.local/lib/python3.6/site-packages/flufl/lock/__init__.py", line 3, in
from flufl.lock._lockfile import (
File "/home/dev2/.local/lib/python3.6/site-packages/flufl/lock/_lockfile.py", line 54, in
public(SEP=SEP)
TypeError: public() got an unexpected keyword argument 'SEP'

$ dvc version

Additional Information (if any):

If applicable, please also provide a --verbose output of the command, eg: dvc add --verbose.

[pared]

@Nikesh46 , how was the dvc installed? pip install dvc?

EDIT:
also, what system do you use?

[Nikesh46]

@pared how was the dvc installed? pip install dvc?

1. yes, through pip install

also, what system do you use?
2.CentOS Linux 7

[efiop]

Hi @Nikesh46 . Could you show pip freeze | grep dvc and pip freeze | grep flufl, please?

[Nikesh46]

@efiop pip3.6 freeze | grep dvc output is -
WARNING: Could not generate requirement for distribution -pex 0.9.10.dev0 (/home/dev2/.local/lib/python3.6/site-packages): Parse error at "'-pex==0.'": Expected W:(abcd...)
dvc==1.0.1
and pip3.6 freeze | grep flufl output is -
WARNING: Could not generate requirement for distribution -pex 0.9.10.dev0 (/home/dev2/.local/lib/python3.6/site-packages): Parse error at "'-pex==0.'": Expected W:(abcd...)
flufl.lock==3.2

[efiop]

@Nikesh46 And could you show pip freeze | grep public, please?

[Nikesh46]

@efiop sure.
pip3.6 freeze | grep public outputs -
WARNING: Could not generate requirement for distribution -pex 0.9.10.dev0 (/home/dev2/.local/lib/python3.6/site-packages): Parse error at "'-pex==0.'": Expected W:(abcd...)
atpublic==1.0
public==2019.4.13
publicsuffix2==2.20191221

[efiop]

@Nikesh46 Looks like atpublic(used by flufl) is conflicting with public package. Take a look at the piptree output to see what is using public package and if you can, uninstall public package. That will do the trick.

[Nikesh46]

@efiop Thanks for your quick suggestions. high number of packages require "public" package so I may be in a stage where I can't uninstall public package. Can you please let me know if there is an alternative to this?

[efiop]

@Nikesh46 Consider installing dvc through snap: sudo snap install dvc --classic, it will isolate dvc in its own environment, so it doesn't conflict with anything. If that doesn't work for you, you could also use our rpm repository or package https://dvc.org/doc/install/linux

[Nikesh46]

@efiop Sure! I will upate my dvc rpm and let you know further information.
Thanks for the help so far.

[Nikesh46]

@efiop
Here are few things I did and the responses I recieved-
➜ command - /usr/bin/dvc pull

+------------------------------------------+
|                                          |
|     Update available 0.94.0 -> 1.0.1     |
|           Run `yum update dvc`           |
|                                          |
+------------------------------------------+
ERROR: unexpected error - module 'gssapi' has no attribute 'exceptions' 
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

Moreover I haven't altered dvc package-
Here's the proof for the same -

➜  temp git:(project_name) ✗ rpm -V dvc 
➜  temp git:(project_name) ✗ echo $?
0

Also, you can find details of dvc package running -
Command - yum info dvc
Output -

Loaded plugins: fastestmirror, langpacks, nvidia
Determining fastest mirrors
 * base: mirrors.viethosting.com
 * centos-sclo-rh: mirror.labkom.id
 * elrepo: ftp.cc.uoc.gr
 * elrepo-kernel: ftp.cc.uoc.gr
 * epel: mirrors.njupt.edu.cn
 * extras: mirror.vanehost.com
 * updates: centos.excellmedia.net
download.opensuse.org_repositories_home_Alexander_Pozdnyakov_CentOS_7_                                                                                           171/171
google-chrome                                                                                                                                                        3/3
ius                                                                                                                                                              679/679
kubernetes                                                                                                                                                       514/514
runner_gitlab-runner                                                                                                                                               70/70
Installed Packages
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments versioning for ML or data science projects

Please do let me know what can be done next

[efiop]

@Nikesh46 Could you show /usr/bin/dvc pull -v, please? Also, if you uninstall dvc from pip, you won't have to use full path to the dvc binary anymore.

[efiop]

Also, looks like we should configure URL and licence in the rpm package... Will do...

[Nikesh46]

/usr/bin/dvc pull -v
I ran the command as suggested and got his output--

ERROR: unexpected error - module 'gssapi' has no attribute 'exceptions'
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/main.py", line 49, in main
  File "dvc/command/data_sync.py", line 32, in run
  File "dvc/repo/__init__.py", line 30, in wrapper
  File "dvc/repo/pull.py", line 30, in pull
  File "dvc/repo/fetch.py", line 53, in _fetch
  File "dvc/data_cloud.py", line 80, in pull
  File "dvc/data_cloud.py", line 31, in get_remote
  File "dvc/data_cloud.py", line 49, in _init_remote
  File "dvc/remote/__init__.py", line 44, in Remote
  File "dvc/remote/ssh/__init__.py", line 53, in __init__
  File "dvc/remote/base.py", line 111, in __init__
  File "dvc/remote/base.py", line 150, in _check_requires
  File "dvc/remote/base.py", line 143, in get_missing_deps
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/__init__.py", line 22, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/transport.py", line 38, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/auth_handler.py", line 72, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/ssh_gss.py", line 58, in <module>
AttributeError: module 'gssapi' has no attribute 'exceptions'
------------------------------------------------------------

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
/usr/bin/dvc pull -v  0.86s user 0.25s system 99% cpu 1.126 total

[efiop]

Added a fix for package info https://github.com/iterative/dvc/pull/4098 .

Looking into that gssapi exception, looks like pyinstaller missed that module.

@Nikesh46 In the mean time I would try sudo snap install dvc --classic or pipx install dvc(https://github.com/pipxproject/pipx), that installs dvc in an isolated environment.

Thank you so much for the feedback! :pray:

[efiop]

@Nikesh46 Hm, not able to reproduce that binary issue. 1.0.1 binaries are now pushed, mind giving them a try?

[Nikesh46]

@Nikesh46 In the mean time I would try sudo snap install dvc --classic or pipx install dvc(https://github.com/pipxproject/pipx), that installs dvc in an isolated environment.

@efiop pipx did the trick. Thank you so much for the help.

[Nikesh46]

@Nikesh46 Hm, not able to reproduce that binary issue. 1.0.1 binaries are now pushed, mind giving them a try?
@efiop
Sure :)

[Nikesh46]

@efiop even after updating dvc I'm still getting the same error-

2020-06-24 12:54:02,298 ERROR: unexpected error - module 'gssapi' has no attribute 'exceptions'
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/main.py", line 53, in main
  File "dvc/command/data_sync.py", line 36, in run
  File "dvc/repo/__init__.py", line 35, in wrapper
  File "dvc/repo/pull.py", line 34, in pull
  File "dvc/repo/fetch.py", line 59, in _fetch
  File "dvc/data_cloud.py", line 82, in pull
  File "dvc/data_cloud.py", line 30, in get_remote
  File "dvc/data_cloud.py", line 48, in _init_remote
  File "dvc/remote/__init__.py", line 83, in get_remote
  File "dvc/remote/__init__.py", line 79, in get_cloud_tree
  File "dvc/remote/ssh/__init__.py", line 53, in __init__
  File "dvc/remote/base.py", line 111, in __init__
  File "dvc/remote/base.py", line 137, in _check_requires
  File "dvc/remote/base.py", line 130, in get_missing_deps
  File "importlib/__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/__init__.py", line 22, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/transport.py", line 38, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/auth_handler.py", line 72, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
  File "site-packages/paramiko/ssh_gss.py", line 58, in <module>
AttributeError: module 'gssapi' has no attribute 'exceptions'

dvc details -

temp git:(project_name) ✗ yum info dvc
Loaded plugins: fastestmirror, langpacks, nvidia
Loading mirror speeds from cached hostfile
 * base: mirrors.viethosting.com
 * centos-sclo-rh: mirror.labkom.id
 * elrepo: ftp.cc.uoc.gr
 * elrepo-kernel: ftp.cc.uoc.gr
 * epel: mirrors.njupt.edu.cn
 * extras: mirror.vanehost.com
 * updates: centos.excellmedia.net
kubernetes                                                                                                                                                       524/524
Installed Packages
Name        : dvc
Arch        : x86_64
Version     : 1.0.1
Release     : 1
Size        : 181 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments versioning for ML or data science projects
yum info dvc  3.31s user 0.39s system 94% cpu 3.926 total

[efiop]

Not able to reproduce in 1.1.3 bin package with or without gss_auth = True :( Closing for now. If anyone else is running into this problem with new versions of dvc - please let us know and we'll reopen.

@Nikesh46 Thanks for the feedback!