Scoutsuite: AWS access credentials not being read from environment

[amak]

According to the documentation, "If credentials have been configured as environment variables, Scout Suite will use these when making API calls".

https://github.com/nccgroup/ScoutSuite/wiki/Amazon-Web-Services

But when I try to use environment variables, Scoutsite CLI insists that I provide the credentials as command line parameters ("error: one of the arguments -p/--profile --access-keys is required"). If I specify "--access-keys", scoutsuite appears to demand the access keys, tokens, etc on the command line.

(This is undesirable, because in my use case scenario, CLI commands and their parameters get persisted in logs, whereas environment variables don't).

Looking through the code, I cannot see any reference to "os.getenv" or "os.environ", where scoutsuite might perhaps be looking in the environment.

I've added such code myself, it's pretty trivial.

So have I just misunderstood the intent? Is documentation maybe a little imprecise or misleading?

If it's a case of needing a patch, I'd be happy to write one.

[x4v13r64]

This seems like a duplicate of https://github.com/nccgroup/ScoutSuite/issues/509 and should be fixed in develop.

[thommor]

@amak It's the boto3 library that uses the environment variables for authentication not ScoutSuite itself as described here.

As @j4v mentioned this should now be fixed in develop if you could try that? Or wait for 5.4 to be released.

[amak]

Yes, the develop branch fixed that problem, thanks.

But it gave rise to new errors, where I got lots of errors like this

"ERROR services.py L55: Could not fetch awslambda configuration: You must specify a region.", for multiple services.

I am definitely specifying a region, with the command line param "--regions", i.e.

scout aws --regions us-west-2 --exclude-regions ap-east-1 me-south-1 --max-workers 4 --no-browser --report-dir scoutsuite-reports

I have also tried setting the environment variable AWS_REGION, but this has not fixed the problem.

When I run it from my laptop, it works fine, because I have my ~/.aws/credentials and ~/.aws/config properly setup, including regions.

But when I try to run it on a CI server, with credentials supplied by "sts assume role", set as environment variables, it fails with the above errors.

[x4v13r64]

That's odd, I'll have to test it out to reproduce. Can you provide exact details on how you're configuring authentication.

[amak]

We use sts to assume a role, e.g. "aws sts assume-role ......."

We then set the following environment variables to contain the values returned by the assume-role call

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN

And the invoke scout with those env vars set, i.e.

scout aws --regions us-west-2 --max-workers 4 --no-browser --report-dir scoutsuite-reports

Then I get the "You must specify a region" error described above.

I tried setting the AWS_REGION env var, but that didn't seem to change anything.

There is no ~/.aws/credentials or ~/.aws/config file.

[x4v13r64]

As per https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#environment-variable-configuration it would be AWS_DEFAULT_REGION (not AWS_REGION), although that still shouldn't be required to run Scout.

I've run it with your described configuration without any issues...

Also are you running the develop branch? I ask because from your CLI params it looks like you're running the pypi build (which is the equivalent of current master branch).

[thommor]

I can reproduce in the develop branch but not been able to track the cause

[thommor]

I think the issue is when you call Session().get_available_services() in ScoutSuite/providers/aws/facade/base.py:L48 as no region is specified and because there is no default region boto can use it fails. Setting AWS_DEFAULT_REGION seems to work

[x4v13r64]

@thommor so is this a Scout issue or a configuration problem?

[thommor]

@j4v I would say a ScoutSuite issue but could be either. ScoutSuite assumes that a default region is specified so if there is no default region specified boto fails.

There are two options:

1. We enforce that a default region is specified for boto either through AWS config or the AWS_DEFAULT_REGION environment variable

2. We provide a default when calling boto. I've tested this and it works and doesn't appear to have any effect on the results (i.e I can have a default region in the code of eu-west-1 and run a scan against us-west-1 and not get any of my EU resources included.

In ScoutSuite/providers/aws/facade/base.py:

I switched from boto.session.Session to boto3.session.Session

Changed L48 from Session().get_available_services() to Session(region_name='eu-west-1').get_available_services()

Changed L52 from Session().get_available_regions(service, partition_name) to Session(region_name='eu-west-1').get_available_regions(service, partition_name)

Changed L55 from self.session.client('ec2').describe_regions(Filters=[{'Name': 'opt-in-status', 'Values': ['not-opted-in']}], AllRegions=True) to self.session.client('ec2', 'eu-west-1').describe_regions(Filters=[{'Name': 'opt-in-status', 'Values': ['not-opted-in']}], AllRegions=True)

Happy to make a PR if you like?

[x4v13r64]

Happy to make a PR if you like?

Sure :smile:

[amak]

Confirmed that my detailed report was from running the develop branch, installed like so

pip install https://github.com/nccgroup/ScoutSuite/archive/develop.zip

[joeylosurdo]

Confirmed that my detailed report was from running the develop branch, installed like so

pip install https://github.com/nccgroup/ScoutSuite/archive/develop.zip

Thanks - works for me. I was experiencing the same issue.

[x4v13r64]

Closing, this has now been fixed in develop.

[amak]

Confirmed that all issues are now fixed in the develop branch, thanks everyone.

(I did have to add permissions to invoke ec2:DescribeRegions to the relevant role)