Scoutsuite: EndpointConnectionError for every AWS endpoint in every region

Created on 2 Jun 2020 · 8Comments · Source: nccgroup/ScoutSuite

I get the below error for every service in every region:

2020-06-02 09:44:47 Alastairs-MBP.play-consult.net scout[7037] ERROR elb.py L65: 
Failed to describe ELB policies: Could not connect to the endpoint URL: 
"https://elasticloadbalancing.us-east-1.amazonaws.com/"

I've run through the troubleshooting steps in https://aws.amazon.com/premiumsupport/knowledge-center/s3-could-not-connect-endpoint-url/ and confirmed that I can resolve the DNS for each of the endpoints correctly, and can ping them successfully.

I'm running python scout.py aws, from a virtualenv-created python3 environment. I received this error when installing requirements, if it's any clue:

ERROR: oci 2.14.3 has requirement cryptography==2.8, but you'll have cryptography 2.9.2
which is incompatible.

My credentials are configured by the AWS_PROFILE environment variable, which result in a role requiring MFA. This part is working correctly, as when I first run Scout I'm asked for my MFA code. The role has AdministratorAccess on the account.

From the stack trace:

 Traceback (most recent call last):
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 752, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/botocore/httpsession.py", line 263, in send
    chunked=self._chunked(request.headers),
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/Users/alastair/Projects/internal/it/compliance/ScoutSuite/.env/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x11c5c9dd0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

I'm happy to provide any more information if it's helpful!

bug potential

Source

alitheg

Most helpful comment

Added https://github.com/nccgroup/ScoutSuite/wiki/Setup#osx

x4v13r64 on 2 Jun 2020

❤2

All 8 comments

Looks like https://github.com/nccgroup/ScoutSuite/issues/719?

x4v13r64 on 2 Jun 2020

Aha! Thank you. I tried to search the errors I was getting but didn't find that one!

I've increased it and progress looks more promising - thank you :)

alitheg on 2 Jun 2020

@alitheg is this a very large AWS account? If we could find a fix for this in the code itself I'd be keen to integrate it.

x4v13r64 on 2 Jun 2020

It's a relatively small account - the report produced when it worked says 264 EC2 resources, 197 IAM, 89 RDS, 75 S3 and 2696 SNS.

I wonder if the default number of file descriptors on Mac is just very small? A quick search suggests that Linux defaults to 1024, whereas my Mac defaults to 256.

alitheg on 2 Jun 2020