Dvc: AWS remote using default profile

Created on 21 Aug 2018  路  4Comments  路  Source: iterative/dvc

DVC Version: 0.17.1
Installation: Mac download

Config:

['remote "s3remote"']
url = s3://<bucket-name>/<dir>/
[core]
remote = s3remote
[aws]
profile = <profile>
storagepath = <bucket-name>/<dir>/
credentialpath = /path/to/.aws/credentials
region = us-east-1

DVC works when I set my default profile's credentials to what's in - so I'm thinking that it's picking up the default profile regardless of what's in the config. It looks like #923 might not be fully resolved?

Most helpful comment

Hi @RowdyHowell !

You are using current and obsoleted syntax together, which results in everything in [aws] not taking any effect, since most of those options should be in ['remote "s3remote"']. The correct config would be:

['remote "s3remote"']
url = s3://<bucket-name>/<dir>/
profile = <profile>
credentialpath = /path/to/.aws/credentials
region = us-east-1
[core]
remote = s3remote

In commands it would look like:

$ dvc remote add s3remote -d s3://<bucket-name>/<dir>/
$ dvc remote modify s3remote profile <profile>
$ dvc remote modify s3remote credentialpath /path/to/.aws/credentials
$ dvc remote modify s3remote region us-east-1

Thanks,
Ruslan

All 4 comments

Hi @RowdyHowell !

You are using current and obsoleted syntax together, which results in everything in [aws] not taking any effect, since most of those options should be in ['remote "s3remote"']. The correct config would be:

['remote "s3remote"']
url = s3://<bucket-name>/<dir>/
profile = <profile>
credentialpath = /path/to/.aws/credentials
region = us-east-1
[core]
remote = s3remote

In commands it would look like:

$ dvc remote add s3remote -d s3://<bucket-name>/<dir>/
$ dvc remote modify s3remote profile <profile>
$ dvc remote modify s3remote credentialpath /path/to/.aws/credentials
$ dvc remote modify s3remote region us-east-1

Thanks,
Ruslan

Perfect, thanks! Do you know where this is documented? I couldn't find where to set the profile before...

Sorry if it was not clear, we will be sure to add a separate article to our user-guide. For now, the most complete description can be found here.

Thanks,
Ruslan

Missed that - that's on me!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

anotherbugmaster picture anotherbugmaster  路  3Comments

shcheklein picture shcheklein  路  3Comments

prihoda picture prihoda  路  3Comments

shcheklein picture shcheklein  路  3Comments

jorgeorpinel picture jorgeorpinel  路  3Comments