Dvc.org: missing information about s3 usage

[churnikov]

Docs don't explicitly say that when one is trying to use s3 or s3 compatible storage there should already exist bucket.

Then, if you would try to push data to s3 and bucket wouldn't exist, you would get the following:

Preparing to upload data to 's3://my/remote/'
Preparing to collect status from s3://my/remote/
[############                  ] 40% Collecting information
ERROR: failed to push data to the cloud - An error occurred (NoSuchBucket) when calling the ListObjectsV2 operation: Unknown

So I propose to explicitly say in docs that you should already have existing bucket to be able to push to s3.
Following docs are related to s3 usage:
https://dvc.org/doc/get-started/configure
https://dvc.org/doc/commands-reference/remote-add
Related discussion in ods:
https://opendatascience.slack.com/archives/CGGLZJ119/p1559831715009900

[pared]

Hi @kuparez! That is a good point! Would you mind creating the issue in our website repository?

[efiop]

Moved to dvc.org repo

[shcheklein]

@efiop @pared we need to improve the error message, guys. Ideally, all remotes should behave the same way (local creates a dir as far as I remember). Documentation should be fixed, yep.

[efiop]

@shcheklein even though error says that NoSuchBucket, but yes, we could catch it and show something friendlier. Created https://github.com/iterative/dvc/issues/2100 .

As to creating dirs, dir is not the same as bucket, we do automatically create prefixes everywhere. But, thinking about it, we actually auto-create containers in Azure, so maybe creating something by-default is fine with s3 too after all. Created https://github.com/iterative/dvc/issues/2101 for that too.

[dnabanita7]

Can I take this up?

[dnabanita7]

Actually, it is mentioned in use-cases to create a bucket before connecting it to dvc remote as of here https://dvc.org/doc/use-cases/share-data-and-model-files . But I think mentioning it in commands-reference/remote and commands-reference/config will do good.

[shcheklein]

@Naba7 definitely, let's mention this!

[shcheklein]

@Naba7 reopening this since it should be added to the documents actually mentioned in the ticket

[dnabanita7]

I should add the same to commands-reference/remote-add. Thanks @shcheklein