Boto3: Boto does not recognize ~.aws/credentials

[ChrisMiami]

Following the Quickstart in the docs and am unable to authenticate using my ~/.aws/credentials files (aws cli commands work OK):

>>> import boto3
>>> 
>>> for bucket in boto3.resource('s3').buckets.all():
...     print(bucket.name)
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/boto3/resources/collection.py", line 83, in __iter__
    for page in self.pages():
  File "/Library/Python/2.7/site-packages/boto3/resources/collection.py", line 161, in pages
    pages = [getattr(client, self._py_operation_name)(**params)]
  File "/Library/Python/2.7/site-packages/botocore/client.py", line 310, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Library/Python/2.7/site-packages/botocore/client.py", line 396, in _make_api_call
    operation_model, request_dict)
  File "/Library/Python/2.7/site-packages/botocore/endpoint.py", line 111, in make_request
    return self._send_request(request_dict, operation_model)
  File "/Library/Python/2.7/site-packages/botocore/endpoint.py", line 136, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/Library/Python/2.7/site-packages/botocore/endpoint.py", line 120, in create_request
    operation_name=operation_model.name)
  File "/Library/Python/2.7/site-packages/botocore/hooks.py", line 226, in emit
    return self._emit(event_name, kwargs)
  File "/Library/Python/2.7/site-packages/botocore/hooks.py", line 209, in _emit
    response = handler(**kwargs)
  File "/Library/Python/2.7/site-packages/botocore/signers.py", line 84, in handler
    return self.sign(operation_name, request)
  File "/Library/Python/2.7/site-packages/botocore/signers.py", line 119, in sign
    signer.add_auth(request=request)
  File "/Library/Python/2.7/site-packages/botocore/auth.py", line 621, in add_auth
    raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials

[kyleknap]

Interesting the CLI and boto3 use roughly the same credential provider. So that should work. Could you do the following things to help debug this:
1) Run configure list in the CLI to see what credentials it is using:

$ aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ******************** shared-credentials-file    
secret_key     ******************** shared-credentials-file    
    region                us-east-1      config-file    ~/.aws/config

2) What version of botocore and boto3 are you using?

[ChrisMiami]

@kyleknap - Here is the output from configure list:

$ aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key                <not set>             None    None
secret_key                <not set>             None    None
    region                us-east-1      config-file    ~/.aws/config

This belies the configuration files that aws configure wrote in the ~/.aws folder it created, which include a named profile and its access/secret keys. Apparently I have a botched aws config?

As far as I know, I'm using boto3 (OS X El Capitan). I had boto installed then uninstalled it and installed boto3, so presumably pip did the right thing:

sudo pip uninstall boto
sudo pip install boto3
sudo pip install boto3 --ignore-installed six

[ChrisMiami]

OK, I re-ran aws configure and copy/pasted the values from the files that were in ~/.aws (config, credentials), and now I've got:

$ aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************EQPQ shared-credentials-file    
secret_key     ****************J7U6 shared-credentials-file    
    region                us-east-1      config-file    ~/.aws/config

Retrying the boto3 quickstart gives me a list of buckets. Thanks! Case closed.

[kyleknap]

Cool. Glad to hear it is now working.

[MarkPare]

I have a similar issue, but I'm able to print bucket names from a python/django shell. Running 'aws configure list' looks good when I do it, but my code fails when trying to use methods like 'generate_presigned_url' in my views. Any suggestions?

[kadrach]

I have a similar issue too. The default "profile" for boto to use is an IAM profile.

Now I need to add an additional (non-default) profile. However boto3/botocore only pickup the ~.aws/credentials if both a default and other_profile are defined.

How do I not specify a default profile, and have an IAM profile be default, while also specifying other other_profile... in the credentials file?

[SeanSnyders]

I have the same situation on boto3 with my [default] profile in the config and credentials file being a IAM user.
AWS CLI works fine, but the boto3 py API in python 2.7.3 (boto3-1.4.1-py2.py3-none-any.whl) does not and seems to not read the config and credentials which is stored at the default locations.

[SeanSnyders]

Scratch my previous comment. I was working in a stale terminal from which I installed AWS in.
New terminal, and all is good.

[rest17]

I am having similar issue. my aws configure list command shows everything correct but on aws ec2 instance on the very first line python script boto3.client('cloudformation') failes with No Region Error.
Turn out when running the pythin with sudo user gives this but with sudo works fines

[scshepard]

Could be an issue with adminuser vice default.

My credentials file has two sections:

[adminuser]

...

[default]

...

---

From: rest17 [e-mail]
Sent: Sunday, March 19, 2017 5:59:17 AM
To: boto/boto3
Cc: Subscribed
Subject: Re: [boto/boto3] Boto does not recognize ~.aws/credentials (#458)

I am having similar issue. my aws configure list command shows everything correct but on aws ec2 instance on the very first line python script boto3.client('cloudformation') failes with No Region Error.
Turn out when running the pythin with sudo user gives this but with sudo works fines

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/boto/boto3/issues/458#issuecomment-287596605, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACKXSwC33qsY6RCJ7LviFgS6DuRBrMVFks5rnMQ1gaJpZM4HMqUf.

[parayaluyanta]

weird in my case,

Had

boto3.setup_default_session(profile_name="aws-federated")                                        
boto3.setup_default_session(region_name='us-west-2')  

had to remove region part and it works

boto3.setup_default_session(profile_name="aws-federated")                                        
##boto3.setup_default_session(region_name='us-west-2')  

[HWBN1]

If you're sure you configure your aws correctly, just make sure the user of the project can read from ./aws or just run your project as a root

[premanandmanimaran]

Hello
I am trying to migrate Rocket chat Data from GridFS to Amazon S3 and I ran into this error
root@ip:~# ./migrate.py -c dump -d /app/uploads -r rocketchat -t AmazonS3 -d bucket_name

1. Dumping iMWFJMgrwKXT9Bm7w Audio record.mp3
   Traceback (most recent call last):
   File "./migrate.py", line 232, in
   obj.dumpfiles("rocketchat_uploads", store)
   File "./migrate.py", line 117, in dumpfiles
   key = store.put(filename, data, upload)
   File "./migrate.py", line 68, in put
   entry['name']) + '"')
   File "/usr/local/lib/python3.6/dist-packages/boto3/resources/factory.py", line 520, in do_action
   response = action(self, *args, kwargs)
   File "/usr/local/lib/python3.6/dist-packages/boto3/resources/action.py", line 83, in __call__
   response = getattr(parent.meta.client, operation_name)(params)
   File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
   return self._make_api_call(operation_name, kwargs)
   File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 648, in _make_api_call
   operation_model, request_dict, request_context)
   File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 667, in _make_request
   return self._endpoint.make_request(operation_model, request_dict)
   File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 102, in make_request
   return self._send_request(request_dict, operation_model)
   File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 132, in _send_request
   request = self.create_request(request_dict, operation_model)
   File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 116, in create_request
   operation_name=operation_model.name)
   File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 356, in emit
   return self._emitter.emit(aliased_event_name, kwargs)
   File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 228, in emit
   return self._emit(event_name, kwargs)
   File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 211, in _emit
   response = handler(kwargs)
   File "/usr/local/lib/python3.6/dist-packages/botocore/signers.py", line 90, in handler
   return self.sign(operation_name, request)
   File "/usr/local/lib/python3.6/dist-packages/botocore/signers.py", line 157, in sign
   auth.add_auth(request)
   File "/usr/local/lib/python3.6/dist-packages/botocore/auth.py", line 425, in add_auth
   super(S3SigV4Auth, self).add_auth(request)
   File "/usr/local/lib/python3.6/dist-packages/botocore/auth.py", line 357, in add_auth
   raise NoCredentialsError
   botocore.exceptions.NoCredentialsError: Unable to locate credentials

Can somebody help