Moto: NoCredentialsError: Unable to locate credentials

[MartinThoma]

See https://stackoverflow.com/q/53220953/562769 / travis log :

I get the error message

botocore.exceptions.NoCredentialsError: Unable to locate credentials

Any idea why?

Environment variables

- AWS_ACCESS_KEY_ID=dummy-access-key
- AWS_SECRET_ACCESS_KEY=dummy-access-key-secret
- AWS_DEFAULT_REGION=us-east-1

Software Versions

• boto: 2.49.0
• boto3: 1.9.41
• botocore: 1.12.41
• moto: 1.3.7

[h-vetinari]

Having the same issue here. Seems the newer boto(core) is more restrictive about the credentials?

[phobologic]

Having this issue as well in my project https://circleci.com/gh/cloudtools/stacker/3235

[sbkg0002]

Same here with the s3 endpoint:

import boto3

BUCKET = 's3mock'
s3 = boto3.resource('s3', use_ssl=False, verify=False)
s3.create_bucket(Bucket=BUCKET)

Traceback (most recent call last):
  File "b", line 5, in <module>
    s3.create_bucket(Bucket=BUCKET)
  File "/usr/lib/python3.6/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/lib/python3.6/site-packages/botocore/client.py", line 320, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3.6/site-packages/botocore/client.py", line 624, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucket operation: The AWS Access Key Id you provided does not exist in our records.

Same goes for boto3.client('s3', use_ssl=False, verify=False)

[whisller]

Seen somewhere (and that helped me) that temporary solution is to downgrade to moto==1.3.6

[alphachai]

What's the status of this issue? I'm running into NoCredentialsError on the latest versions of everything. Issues #1793 and #1796 seem to be essentially the same thing but are marked closed with merge of #1907.

I've tried freezing various other combinations of requirements without luck, so far.

moto 1.3.7
boto3 1.9.67
botocore 1.12.67

AWS_ACCESS_KEY_ID=foobar
AWS_SECRET_ACCESS_KEY=foobar



md5-6441a5ee9fbeae92c9f2706ad88aa3bc



@mock_kinesis
def test_moto_kinesis(self):
    client = boto3.client('kinesis', region_name=settings.AWS_REGION)
    client.put_record(
        StreamName='foobar',
        Data=json.dumps({'foo': 'bar'}),
        PartitionKey='foobar'
    )



md5-6441a5ee9fbeae92c9f2706ad88aa3bc



.venv/lib/python3.6/site-packages/botocore/client.py:320: in _api_call
    return self._make_api_call(operation_name, kwargs)
.venv/lib/python3.6/site-packages/botocore/client.py:611: in _make_api_call
    operation_model, request_dict)
.venv/lib/python3.6/site-packages/botocore/endpoint.py:102: in make_request
    return self._send_request(request_dict, operation_model)
.venv/lib/python3.6/site-packages/botocore/endpoint.py:132: in _send_request
    request = self.create_request(request_dict, operation_model)
.venv/lib/python3.6/site-packages/botocore/endpoint.py:116: in create_request
    operation_name=operation_model.name)
.venv/lib/python3.6/site-packages/botocore/hooks.py:356: in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
.venv/lib/python3.6/site-packages/botocore/hooks.py:228: in emit
    return self._emit(event_name, kwargs)
.venv/lib/python3.6/site-packages/botocore/hooks.py:211: in _emit
    response = handler(**kwargs)
.venv/lib/python3.6/site-packages/botocore/signers.py:90: in handler
    return self.sign(operation_name, request)
.venv/lib/python3.6/site-packages/botocore/signers.py:157: in sign
    auth.add_auth(request)
_ _ _ _ _
self = <botocore.auth.SigV4Auth object at 0x1179d91d0>, request = <botocore.awsrequest.AWSRequest object at 0x114d7d828>

    def add_auth(self, request):
        if self.credentials is None:
>           raise NoCredentialsError
E           botocore.exceptions.NoCredentialsError: Unable to locate credentials

Happy to contribute, but curious if someone is already working on this.

edit pinning boto3 = "~=1.6" and moto = "~=1.3” is my workaround right now.

[dargueta]

Try adding the environment variable BOTO_CONFIG=/dev/null. I currently have

boto3==1.9.71
botocore==1.12.71
moto==1.3.7

and it works fine. The combination of all of these also seems to have fixed #1596 somehow.

[spulec]

Should be fixed with https://github.com/spulec/moto/pull/1952

Feel free to reopen if there are still issues

[3h4x]

@spulec
It's still an issue. Do you need more information about this, like whole pip freeze?

boto3==1.9.71
botocore==1.12.86
moto==1.3.7

Simple test pushing messages

    @mock_sqs
    def test_push_to_queue(self):
        client = boto3.client('sqs')
        client.send_message_batch(
            QueueUrl='',
            Entries=[{'Id': '0', 'MessageBody': '0'}],
        )

botocore.exceptions.NoCredentialsError: Unable to locate credentials

[dargueta]

@3h4x there hasn't been a release with this change yet. If you install moto from master it'll work (at least it does for me?). If you can't do that, you'll have to wait until the next version is released.

[3h4x]

Thanks @dargueta , that indeed fixed my problem.
Let me share quick pip command for people that will stumble upon this issue:
pip install -e git+https://github.com/spulec/moto@master#egg=moto

[aonamrata]

@spulec Can you please create a release with latest changes from master?

[MacHu-GWU]

I solved this by adding a script and a dummy credential file. Then just do from my_package.tests import moto_compat before you import moto:

You can find the solution example in my project:

• set up: https://github.com/MacHu-GWU/s3iotools-project/blob/master/s3iotools/tests/moto_compat.py
• usage: https://github.com/MacHu-GWU/s3iotools-project/blob/master/tests/io/test_dataframe.py#L8

<my_package>
|--- tests
    |--- credentials
    |--- moto_compat.py

content of credentials:

[default]
{field1} = AAAAAAAAAAAAAAAAAAAA
{field2} = AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

content of moto_compat.py:

# -*- coding: utf-8 -*-

import os

src = os.path.join(os.path.dirname(__file__), "credentials")
dst = os.path.join(os.path.expanduser("~"), ".aws", "credentials")
dst_dir = os.path.dirname(dst)

if not os.path.exists(dst_dir):
    os.mkdir(dst_dir)
if not os.path.exists(dst):
    with open(src, "rb") as f1:
        with open(dst, "wb") as f2:
            s = f1.read().decode("utf-8").format(field1="aws_access_key_id", field2="aws_secret_access_key") # this is to fool git-secret
            f2.write(s.encode("utf-8"))

[NeuroWinter]

I am still getting the following error when trying to donwload form s3:

botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the GetObject operation: The AWS Access Key Id you provided does not exist in our records.

This is the line that is causing the error in my tests:
obj = s3_client.get_object(Bucket=bucket_name, Key=f"{location}{file_name}")

Here is the full test:

    @mock_s3
    def test_upload_data_to_s3(self):
        """
        Test to ensure that uploading to s3 is working.
        """
        bucket_name = "testbucket"
        location    = "/TEST/"
        file_name   = "test.txt"
        data        = "THIS IS DATA!"
        s3_client   = boto3.client('s3')
        s3_client.create_bucket(Bucket=bucket_name)
        # We need to create the bucket since this is all in Moto's 'virtual' AWS account
        upload_data_to_s3(bucket_name, location, file_name, data)
        # Now we need to read from the bucket to make sure the data is the same.
        obj  = s3_client.get_object(Bucket=bucket_name, Key=f"{location}{file_name}")

I have tried to add these to my environment:

export BOTO_CONFIG=/dev/null
export AWS_SECRET_ACCESS_KEY=foobar_secret
export AWS_ACCESS_KEY_ID=foobar_key 

This is my boto version:

boto==2.49.0
boto3==1.9.115
botocore==1.12.115

and I installed moto via this command from above:

pip install -e git+https://github.com/spulec/moto@master#egg=moto

[ekuusela]

I got the same InvalidAccessKeyId error but turns out I had a mistake in my configuration. It caused an extra / in file keys. Eg. bucket/filename was mistakenly written out as bucket//filename.

This wasn't a problem with moto in server mode, but the regular mocked s3 didn't work.

[staticfox]

Can this issue be re-opened? I ran in to the same issue as OP and had to spend quiet a bit of time digging through this project's issues in order to get the example to work. Unsure why it was closed when the fix has not been released and nothing in the readme mentions the requirement of pulling from master.

[aonamrata]

I think what we need is a new release from master. Last was on on Nov 5, 2018 but fix was merged sometime in Feb 2019
@spulec ^

[JarnoRFB]

The error botocore.exceptions.NoCredentialsError: Unable to locate credentials popped up for me when going from 1.3.13 to 1.3.14.

[medley56]

Confirming @JarnoRFB's comment. We've pinned moto==1.3.13 and our tests pass again. This is a super annoying breaking change because no one wants to copy AWS credentials files into testing docker containers, nor should it be necessary.

[plbenpeterson]

We have the same issue, pinned our tests to 1.3.13 and they're passing again.

[spulec]

Can you test against the master branch? I'm curious if this was fixed by https://github.com/spulec/moto/pull/2578

[plbenpeterson]

I just now (5 minutes ago) installed moto via pip and received version 1.3.14 - that version fails. Is that what you mean by testing against the master branch? Sorry if I missed something

[medley56]

@plbenpeterson he meant install directly from the head of the master branch, which presumably contains a bugfix that isn't in the most recent version. i.e. use
pip install git+https://github.com/spulec/moto.git@master

[medley56]

@spulec Following up. I tried this and the issue persists in master.

[plbenpeterson]

Thanks for the clarification and follow up. I’m actually using this in pipenv, not installing directly.

Sent from my iPhone

On Nov 21, 2019, at 12:48 PM, Gavin Medley <[email protected][e-mail]> wrote:

[External Email]

Following up. I tried this and the bug persists in master.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/spulec/moto/issues/1941?email_source=notifications&email_token=AD6UAB5HL32BH6ZHPPRPNUTQU3XZHA5CNFSM4GCY632KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE3TFVQ#issuecomment-557265622, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD6UAB2JXPKIPWG5NHPRYTDQU3XZHANCNFSM4GCY632A.

==============================================================================

The information in this e-mail and any attachments are for the sole use of the intended recipient and may contain privileged or confidential information. Delivery to other than the intended recipient shall not be deemed to waive any privilege. Any unauthorized use, disclosure, copying or distribution of this message or attachment is strictly prohibited. If you believe that you have received this e-mail in error, please contact the sender immediately and delete the e-mail and all of its attachments.

[medley56]

@plbenpeterson This is a bit tangential but you can still install packages from git using pipenv. See "Installing from Git:" in the pipenv main readme page: https://github.com/pypa/pipenv

[spulec]

I think I found the issue. This looks like a new regression with https://github.com/spulec/moto/pull/2285/commits/79cd1e609c5ff7153e11956c1f164cda504732fe

If you set up your client before the mock starts, the client won't have any credentials and changing the environment variables at that point won't help (the client doesn't recheck them).

The change with https://github.com/spulec/moto/pull/2578/ might start to make a fix easier, but I think this is still going to be complicated. Any client that has been created will be pointing to the existing boto3 DEFAULT_SESSION. We can force that session to refresh the credentials, but that will have side-effects which persist after we are done mocking.

As a workaround for now, if you set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as environment variables to anything, the tests should pass.

[medley56]

We're just pinning 1.3.13 until your next release. Thanks for addressing this!

[mgcos1231]

It happens again from v1.3.14

I have following version.
boto3==1.11.14
botocore==1.14.14
moto==1.3.14

I followed the workaround suggestion from sulec. it helps.

os.environ["AWS_ACCESS_KEY_ID"] = "test"
os.environ["AWS_SECRET_ACCESS_KEY"] = "test"

[Rackedup22193]

It happens again from v1.3.14

I have following version.
boto3==1.11.14
botocore==1.14.14
moto==1.3.14

I followed the workaround suggestion from sulec. it helps.

os.environ["AWS_ACCESS_KEY_ID"] = "test"
os.environ["AWS_SECRET_ACCESS_KEY"] = "test"

mgcos1231
Thanks this actually worked for me.

[ijoseph]

Workaround works but boy, is it ugly.

os.environ["AWS_ACCESS_KEY_ID"] = "test"
os.environ["AWS_SECRET_ACCESS_KEY"] = "test"

[sonphnt]

What is the status of this issue, guys? Still using workaround way with a couple lines of "os.environ" ?

[Zaubeerer]

I still get the following error message when running the example code:

botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the GetSecretValue operation: The security token included in the request is invalid.

This is super frustrating :(

[JarnoRFB]

This seems to be desired behavior now and is even documented in the README https://github.com/spulec/moto#very-important----recommended-usage

[alfredocambera]

I'm using version 1.13.16 and exporting the environment variables hasn't worked for me.

[siruku6]

I also got the error botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) on travis.ci.
Despite being successful on local PC at that time.

But I followed moto's README.md, and then I was able to avoid the error.

Very Important -- Recommended Usage
Have you tried it already?

---

Environment

|module|version|
|-----------|----------|
|moto |1.3.16 |
|boto3 |1.16.40 |
|botocore |1.19.40 |

[ommmr]

Also getting the same problem. Unable to locate credentials even though they are in settings.py. Have latest version of moto boto3 and botocore

[siruku6]

@ommmr
Could you locate and read the other environment values in settings.py ?
Did you try to locate your values directly into OS or CI tool ?

Perhaps, you could not locate them, but if located, then it may be solved (I'm expecting ...).