Moto: Issue with basic example

[lhucinequr]

Hello,
I am using a Boto3 in a project of mine, I was trying to do my testing using moto but then while trying to do the basic test presented in the documentation I am having the following error:
E botocore.exceptions.NoCredentialsError: Unable to locate credentials.
It seems like moto is not mocking anything

The example I used is the following

import boto3
from moto import mock_s3
from mymodule import MyModel
@mock_s3
def test_my_model_save():
    conn = boto3.resource('s3', region_name='us-east-1')
    # We need to create the bucket since this is all in Moto's 'virtual' AWS account
    conn.create_bucket(Bucket='mybucket')
    model_instance = MyModel('steve', 'is awesome')
    model_instance.save()
    body = conn.Object('mybucket', 'steve').get()['Body'].read().decode("utf-8"
    assert body == b'is awesome'

[lhucinequr]

Just a follow up, the same test above works in the following environment

boto==2.49.0
boto3==1.7.84
botocore==1.10.84

But not with latest releases of each of the packages

[michael-k]

We have nearly the same exception. I've provided stacktraces below. But they are not very useful as the code around line 612 in botocore/client.py was last changed a few years ago.

The breaking release is botocore 1.11.0.

• changelog: https://github.com/boto/botocore/blob/1.11.0/CHANGELOG.rst#1110
• upgrade notes: https://botocore.amazonaws.com/v1/documentation/api/latest/index.html#upgrading-to-1-11-0

With fake credentials:

/usr/local/lib/python3.6/site-packages/boto3/resources/factory.py:520: in do_action
    response = action(self, *args, **kwargs)
/usr/local/lib/python3.6/site-packages/boto3/resources/action.py:83: in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
/usr/local/lib/python3.6/site-packages/botocore/client.py:314: in _api_call
    return self._make_api_call(operation_name, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <botocore.client.S3 object at 0x7f8279278240>, operation_name = 'PutObject', api_params = {'Body': <_io.BytesIO object at 0x7f825a53b518>, 'Bucket': 'TEST-bucket', 'Key': 'test_aggregator/base/0.json.gz'}

    def _make_api_call(self, operation_name, api_params):
        operation_model = self._service_model.operation_model(operation_name)
        service_name = self._service_model.service_name
        history_recorder.record('API_CALL', {
            'service': service_name,
            'operation': operation_name,
            'params': api_params,
        })
        if operation_model.deprecated:
            logger.debug('Warning: %s.%s() is deprecated',
                         service_name, operation_name)
        request_context = {
            'client_region': self.meta.region_name,
            'client_config': self.meta.config,
            'has_streaming_input': operation_model.has_streaming_input,
            'auth_type': operation_model.auth_type,
        }
        request_dict = self._convert_to_request_dict(
            api_params, operation_model, context=request_context)

        handler, event_response = self.meta.events.emit_until_response(
            'before-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            model=operation_model, params=request_dict,
            request_signer=self._request_signer, context=request_context)

        if event_response is not None:
            http, parsed_response = event_response
        else:
            http, parsed_response = self._endpoint.make_request(
                operation_model, request_dict)

        self.meta.events.emit(
            'after-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            http_response=http, parsed=parsed_response,
            model=operation_model, context=request_context
        )

        if http.status_code >= 300:
            error_code = parsed_response.get("Error", {}).get("Code")
            error_class = self.exceptions.from_code(error_code)
>           raise error_class(parsed_response, operation_name)
E           botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.

/usr/local/lib/python3.6/site-packages/botocore/client.py:612: ClientError

With empty credentials (omitted part is identical to above):

        if http.status_code >= 300:
            error_code = parsed_response.get("Error", {}).get("Code")
            error_class = self.exceptions.from_code(error_code)
>           raise error_class(parsed_response, operation_name)
E           botocore.exceptions.ClientError: An error occurred (AuthorizationHeaderMalformed) when calling the PutObject operation: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.
/usr/local/lib/python3.6/site-packages/botocore/client.py:612: ClientError

[lhucinequr]

I think that you're right, it is the latest version of botocore that breaking moto.
By the way when doing pip install moto here is what is installed by default

boto==2.49.0
boto3==1.8.2
botocore==1.11.2

I will be using an earlier release for the time being.
I can't believe that I spent an entire day for this. took me a long time to realize

[ghost]

Same here:

$ AWS_ACCESS_KEY_ID='1111' AWS_SECRET_ACCESS_KEY='2222' python repro.py 
Traceback (most recent call last):
  File "repro.py", line 12, in <module>
    main()
  File "/home/sbv/.local/share/virtualenvs/python-jD43VpKQ/lib/python3.6/site-packages/moto/core/models.py", line 71, in wrapper
    result = func(*args, **kwargs)
  File "repro.py", line 8, in main
    s3.create_bucket(Bucket='test1111')
  File "/home/sbv/.local/share/virtualenvs/python-jD43VpKQ/lib/python3.6/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/home/sbv/.local/share/virtualenvs/python-jD43VpKQ/lib/python3.6/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/home/sbv/.local/share/virtualenvs/python-jD43VpKQ/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/sbv/.local/share/virtualenvs/python-jD43VpKQ/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucket operation: The AWS Access Key Id you provided does not exist in our records.

$ pip freeze
....
boto==2.49.0
boto3==1.8.2
botocore==1.11.2
moto==1.3.4
....



md5-6e6858ff85b6b84e3e6ea1ab526b2216



import boto3
from moto import mock_s3

@mock_s3
def main():
    s3 = boto3.resource('s3', region_name='eu-central-1')

    s3.create_bucket(Bucket='test1111')


if __name__ == '__main__':
    main()



md5-d3e1adcb3ae5268a4ff156c4f51882ad



boto==2.49.0
boto3==1.7.47
botocore==1.10.72
moto==1.3.4

[fnubalaj]

We are also seeing the same issue.

Traceback (most recent call last):
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/moto/core/models.py", line 71, in wrapper
result = func(args, kwargs)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/moto/core/models.py", line 71, in wrapper
result = func(args, *kwargs)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/moto/core/models.py", line 71, in wrapper
result = func(args, **kwargs)
File "cli/tests/cfncluster-unittest.py", line 129, in test_cfn_cluster_create_wait
template_url = setup_configurations()
File "cli/tests/cfncluster-unittest.py", line 46, in setup_configurations
s3.create_bucket(Bucket='us-east-1-cfncluster')
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/client.py", line 599, in _make_api_call
operation_model, request_dict)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/endpoint.py", line 131, in _send_request
request = self.create_request(request_dict, operation_model)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/endpoint.py", line 115, in create_request
operation_name=operation_model.name)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/hooks.py", line 360, in _emit
aliased_event_name, kwargs, stop_on_response
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(kwargs)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/signers.py", line 90, in handler
return self.sign(operation_name, request)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/signers.py", line 156, in sign
auth.add_auth(request)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/auth.py", line 420, in add_auth
super(S3SigV4Auth, self).add_auth(request)
File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/botocore/auth.py", line 352, in add_auth
raise NoCredentialsError
NoCredentialsError: Unable to locate credentials

This was working with Boto3 1.7.x

[meneal]

If #509 is merged will a new version be cut?

[ghost]

And this issue seems to be a duplicate of #1793

[spulec]

Closing as a duplicate of #1793