Sops: AWS_PROFILE not respected with aws credentials as environment variables

[SamGallagher95]

The Issue

When you have an aws config file setup with a profile, and the credentials are stored as environment variables, sops will ignore the profile and attempt to use the default user instead.

How to reproduce

The aws config is setup with a specific profile

~/.aws/config ---

[default]
region = us-east-1

[profile testing]
role_arn = {ROLE_ARN}
credential_source = Environment

And you have these environment variables set

AWS_ACCESS_KEY_ID={KEY}
AWS_SECRET_ACCESS_KEY={SECRET_KEY}
AWS_PROFILE=testing
AWS_SDK_LOAD_CONFIG=1

The expected behavior would be to use the profile testing to authenticate with aws and get the kms key. What actually happens is sops uses the default user associated with the access_key and secret_key to get the key, which fails.

Workaround

To use the AWS_PROFILE I had to do the following.

I used the aws cli to generate a ~/.aws/credentials file. I also had to unset the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables.

I had to edit the ~/.aws/config file from credential_source = Environment to source_profile = default.

Ideal Solution

I would like to be able to use environment variables for my aws credentials with a AWS_PROFILE variable set. Needing to build out the credentials file and unset the other variables is extremely cumbersome when I am automating this process.

[autrilla]

This seems to me like an issue with the AWS Go SDK and not with sops. We don't really do anything with AWS_PROFILE, unless you pass --aws-profile as a flag, in which case we force usage of that profile here:

https://github.com/mozilla/sops/blob/83a354e92fd5d0be1d5bdcfe7cf0b7f68f954ed9/kms/keysource.go#L198-L200

[SamGallagher95]

@autrilla I appreciate the quick response!

I agree that this possibly is an issue with the AWS Go SDK, I just seem to be running into it when using sops.

Interesting note: That --aws-profile does nothing for me. I've tested using that to pass in the name of the aws profile I want to use, but it has the same behavior as not using it.

The only way I can successfully use an aws profile is to unset EITHER the AWS_ACCESS_KEY_ID or the AWS_SECRET_ACCESS_KEY.

[autrilla]

Apparently --aws-profile is broken too: #439. We haven't really changed that code in a while, so it might be something in AWS's side. I'm not sure I'm going to have time to look into it, so marking it as help wanted...

[richstokes]

Could this at all be related to https://github.com/mozilla/sops/issues/614 ? As that is giving me a really weird issue realted to --aws-profile

[ajvb]

This should be partially resolved in the new release of sops, as we were not respecting the ~/.aws/config file in former versions

[sc250024]

To be honest, after the v3.6.0 release, this is still an issue. I'm using Sops v3.6.0, with credentials obtained via gimme-aws-creds through Okta. This means that the ~/.aws/credentials file uses temporary aws_access_key_id and aws_secret_access_key keys.

Creating a new file results in the following error:

$ AWS_PROFILE=development sops test.yaml

Results in the following error:

$ sops --verbose test.yaml

[AWSKMS]     INFO[0005] Encryption failed                             arn="arn:aws:kms:us-east-1:111122223333:key/11111c1d-b2c8-437d-ae4b-d00a123ccc45"
Error encrypting the data key with one or more master keys: [failed to encrypt new data key with master key "arn:aws:kms:us-east-1:111122223333:key/11111c1d-b2c8-437d-ae4b-d00a123ccc45": Failed to call KMS encryption service: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors]

But I can use the key just fine with the AWS cli:

$ AWS_PROFILE=development aws kms encrypt \
--key-id 11111c1d-b2c8-437d-ae4b-d00a123ccc45 \
--plaintext my-secret-text \
--query CiphertextBlob \
--output text

So yeah.... a bit annoying.

[yakhyadabo]

Hi,
Any update regarding this issue ?

[sc250024]

Hi,
Any update regarding this issue ?

Have you updated to 3.6.1? The AWS SDK was bumped, and it should solve this issue.

[yakhyadabo]

I've upgraded to 3.6.1 and it works.
Thanks!

[autrilla]

I guess we forgot to close it :)