Dvc: unexpected error - invalid literal for int() with base 10: '\\dvc_repo'

[shawnlee103]

Bug Report

DVC version: 1.2.2
Python version: 3.6.9
Platform: Linux-5.4.0-42-generic-x86_64-with-Ubuntu-18.04-bionic
Binary: False
Package: pip
Supported remotes: http, https, ssh
Cache: reflink - not supported, hardlink - supported, symlink - supported
Filesystem type (cache directory): ('ext4', '/dev/sda2')
Repo: dvc, git
Filesystem type (workspace): ('ext4', '/dev/sda2')

due to NAS storage, i have to set SSH path with "\"

['remote "data_ssh"']
url = ssh://[email protected]:dvc_repo
ask_password = true

when dvc push

2020-07-30 16:54:23,655 DEBUG: fetched: [(122050,)]
2020-07-30 16:54:23,663 ERROR: unexpected error - invalid literal for int() with base 10: '\\dvc_repo'
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/main.py", line 53, in main
    ret = cmd.run()
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/command/data_sync.py", line 59, in run
    run_cache=self.args.run_cache,
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/repo/__init__.py", line 34, in wrapper
    ret = f(repo, *args, **kwargs)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/repo/push.py", line 35, in push
    return len(used_run_cache) + self.cloud.push(used, jobs, remote=remote)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/data_cloud.py", line 63, in push
    remote = self.get_remote(remote, "push")
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/data_cloud.py", line 30, in get_remote
    return self._init_remote(name)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/data_cloud.py", line 48, in _init_remote
    return get_remote(self.repo, name=name)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/remote/__init__.py", line 8, in get_remote
    tree = get_cloud_tree(repo, **kwargs)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/tree/__init__.py", line 78, in get_cloud_tree
    return _get_tree(remote_conf)(repo, remote_conf)
  File "/home/XXX/.local/lib/python3.6/site-packages/dvc/tree/ssh/__init__.py", line 64, in __init__
    or parsed.port
  File "/usr/lib/python3.6/urllib/parse.py", line 169, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: '\\dvc_repo'
------------------------------------------------------------

[efiop]

Hi @shawnlee103 !

Are you able to use that url with things like scp or ssh?

[shawnlee103]

Hi @efiop ,

Yes. I try
scp file.data [email protected]:dvc_repo
scp file2.data [email protected]:dvc_repo/dir1/dir2
It can work all.

[efiop]

@shawnlee103 Whoa, so even mixing the slashes works? O_o And so it puts file2.data into something like C:\\dvc_repo\dir1\dir2 ?

[efiop]

Looking at that path, I would expect your NAS's sftp server to be something custom that can handle weird stuff like that, but we need to double check the sftp spec. Any details about the NAS, btw? Anything special?

Are you sure it requies and won't work with normal paths?

[shawnlee103]

Hi efiop,

I used synology nas.
scp file2.data [email protected]:dvc_repo/dir1/dir2
scp file2.data [email protected]:dvc_repo/dir1/dir2
it can work all.
scp file2.data [email protected]:/dvc_repo/dir1/dir2
it does't work.(scp: /dvc_repo/dir1/dir2: No such file or directory)

[core]
remote = data_ssh
['remote "data_ssh"']
url = ssh://[email protected]:dvc_repo
ask_password = true

after 'dvc push'
ERROR: unexpected error - invalid literal for int() with base 10: 'dvc_repo'

[pmrowla]

Testing this locally with a synology NAS, I was not able to reproduce this scenario, my particular device (DS418j running DSM 6.2.3-25426) rejects scp paths with the leading backslash.

In any case, it looks like the actual issue is the same as #4167. In DVC if your SSH remote URL contains a : we interpret what comes after the colon as a port number, not a relative path.

Since we don't support relative paths in our SSH remote URL scheme, you will need to work around it by specifiying the full path. On synology devices, the full path should be /volumeN/... where N is the storage volume number.

@shawnlee103 can you try configuring DVC to use ssh://user@ip/volume1/dvc_repo for the remote URL? (If your dvc_repo shared folder is on a different volume, you will need to use the correct number).

[shawnlee103]

Hi All,

ssh://user@ip/volume1/dvc_repo
not work.

---

But
[email protected]:/dvc_repo/dir1
It can work after i change user root directory to home directory on NAS.
Thanks.
I will close it.
Thank you for your help.