I have been attempting to use dvc to keep track of my files. This was my first pass at trying to get something to work.
Commands I have ran
$ dvc add data/<my-data>
...
$ dvc remote add -d digital-ocean https://<my space>.nyc3.digitaloceanspaces.com/<my base path>
I have attempted to add an endpointurl as specified in https://dvc.org/doc/command-reference/remote/add#supported-storage-types under s3 compatible storage, but that causes a different error. Specifically it caused
ERROR: configuration error - config file error: extra keys not allowed @ data['remote']['digital-ocean']['endpointurl']
When I set my url to be https://<my space>.nyc3.digitaloceanspaces.com/<my base path> and withhold setting an endpointurl, I end up getting
2020-08-09 09:19:17,823 DEBUG: Uploading '.dvc/cache/<file>' to 'https://<my space>.nyc3.digitaloceanspaces.com/<my bucket path>/<file>'
2020-08-09 09:19:17,832 ERROR: failed to upload '.dvc/cache/<file>' to 'https://<my space>.nyc3.digitaloceanspaces.com/<my bucket path>/<file>' - '411 Length Required'
for every single file I am attempting to upload. When I look to see the size of the files in my cache, they all appear to be non-empty.
$ dvc version
DVC version: 1.3.1 (osxpkg)
---------------------------------
Platform: Python 3.7.5 on Darwin-19.6.0-x86_64-i386-64bit
Supports: All remotes
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s5
Workspace directory: apfs on /dev/disk1s5
Repo: dvc, git
@desa hi! for S3 (and S3-compatible), remote storage URL starts with s3 prefix. Please try something like this:
dvc remote add -d digital-ocean s3://my-bucket/path-to-storage
dvc remote modify digital-ocean endpointurl https://nyc3.digitaloceanspaces.com
Thanks for getting back to me! That totally worked. Thanks so much. Sorry for opening an unnecessary issue. 馃憤
Most helpful comment
@desa hi! for S3 (and S3-compatible), remote storage URL starts with s3 prefix. Please try something like this: