Scoutsuite: Using botocore>=1.12.136 prints multiple AuthFailure errors

[gebailey]

• pip install -U botocore==1.12.135 boto3==1.9.135

Normal ScoutSuite run

• pip install -U botocore==1.12.136 boto3==1.9.136

Results in numerous warnings/errors, such as:

botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the DescribeAlarms operation: The security token included in the request is invalid.
Task exception was never retrieved
future: <Task finished coro=<Stacks.fetch_all() done, defined at /home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/resources/cloudformation/base.py:13> exception=ClientError('An error occurred (InvalidClientTokenId) when calling the ListStacks operation: The security token included in the request is invalid.',)>
Traceback (most recent call last):
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/resources/cloudformation/base.py", line 14, in fetch_all
    raw_stacks = await self.facade.cloudformation.get_stacks(self.region)
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/facade/cloudformation.py", line 14, in get_stacks
    'cloudformation', region, self.session, 'list_stacks', 'StackSummaries')
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/facade/utils.py", line 28, in get_all_pages
    service, region, session, paginator_name, [entity], **paginator_args)
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/facade/utils.py", line 58, in get_multiple_entities_from_all_pages
    return await run_concurrently(lambda: AWSFacadeUtils._get_all_pages_from_paginator(paginator, entities))
  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/facade/utils.py", line 58, in <lambda>
    return await run_concurrently(lambda: AWSFacadeUtils._get_all_pages_from_paginator(paginator, entities))
  File "/home/gbailey/git/ScoutSuite/ScoutSuite/providers/aws/facade/utils.py", line 74, in _get_all_pages_from_paginator
    for page in paginator:
  File "/home/gbailey/ss50/lib64/python3.6/site-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/home/gbailey/ss50/lib64/python3.6/site-packages/botocore/paginate.py", line 332, in _make_request
    return self._method(**current_kwargs)
  File "/home/gbailey/ss50/lib64/python3.6/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/gbailey/ss50/lib64/python3.6/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)

[x4v13r64]

This is likely caused by https://github.com/boto/boto3/issues/1943 - specifying a region other then ap-east-1 allows scans to complete.

[rtomlinson-latacora]

We're running into a similar issue with the auth errors.
boto3 - 1.9.137
botocore - 1.12.137
This is even after the ap-east-1 region has been enabled. We're finding that describe-regions on the cli still doesn't report ap-east-1 as available if you specify a region other than ap-east-1.

[rtomlinson-latacora]

Scoutsuite seems to run fine as long as all the regions are enabled.

@j4v Is there a way to specify multiple regions for a scan? I'm assuming that we lose the ability to have a single, cross-region report if we specify the region.

[x4v13r64]

Scoutsuite seems to run fine as long as all the regions are enabled.

Gotcha, hopefully they will resolve this in boto ASAP.

Is there a way to specify multiple regions for a scan?

Sure, just specify --regions eu-west-1 eu-west-2 <more regions>

[stormfleet]

I'll tag onto this, still appears to be an issue doesn't it?

[x4v13r64]

@stormfleet yes, tracked in https://github.com/boto/boto3/issues/2022 but they have yet to provide a solution.

[x4v13r64]

develop now has a --exclude-regions parameter. This can be used to exclude optional regions, e.g. by passing --exclude-regions ap-east-1 me-south-1.

edit this is now merged into master / latest releases

[aph3rson]

I mentioned this in boto/boto3#2022, but I'm including it here for reference:

FYI, it looks like there's been a recent (last month or so?) update to this. Per Managing AWS Regions:

Describing Your Regions Using the AWS CLI

Use the describe-regions command to describe the Regions available for your account, whether they are enabled or disabled.
aws ec2 describe-regions --all-regions

If the Region is enabled by default, the output includes the following:
"OptInStatus": "opt-in-not-required"

If the Region is not enabled, the output includes the following:
"OptInStatus": "not-opted-in"

After an opt-in Region is enabled, the output includes the following:
"OptInStatus": "opted-in"

Ergo, as the shape of DescribeRegions has changed, this should be something that can be detected when creating the boto session. Note that I'm here from nccgroup/ScoutSuite#381, so I'm not sure if it would be logic that would be in a boto client, or in boto/botocore.

[x4v13r64]

Thanks @aph3rson, leveraged that to implement a fix.

Closing now as implemented in develop.

[r0y1or1]

does 5.4.0 includes develop ?

[x4v13r64]

@r0y1or1 yes, this has been merged into master and the latest release (5.4).

If you're running into auth errors with 5.4 it's unlikely to be the same issue.

[r0y1or1]

i ran 5.4.0, seems to be fixed. thanks !

[x4v13r64]

Reopening to validate if https://github.com/boto/boto3/issues/1943#issuecomment-645630450 can provide a more elegant fix.