Vault: AWS IAM authentication issues after upgrading from 1.5.2 from 1.5.0

[fernando-villalba]

We are getting the following errors after upgrading to vault 1.5.2 from 1.5.0

Cannot login using AWS-IAM: invalid request header: Host
and

Cannot login using AWS-IAM: invalid request header: X-Amz-Security-Token; nested exception is org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request org.springframework.vault.authentication.VaultLoginException: Cannot login using AWS-IAM: invalid request header: X-Amz-Security-Token;

I have absolutely no idea why this is happening or what I can do to fix it. I just know it happens after upgrading to 1.5.2 from 1.5.0 and reverting to 1.5.0 fixes the issue.

I know the latest version 1.5.2 included a security fix for vault AWS IAM authentication but I wasn't able to find enough information for me to figure out if I need change any config after this upgrade in your upgrade guide or anywhere else.

If more information is needed let me know and maybe I can provide it.

[kalafut]

@fernando-villalba Thank you for the report! Yes, the security update did add some extra header checking, and you've found an issue with our default accept list, which we'll fix. Fortunately you can specify the allowed header list explicitly, though you'll need to include them all. To set the current defaults + [host, x-amz-security-token] will look like:

vault write auth/aws/config/client allowed_sts_header_values="Content-Type" allowed_sts_header_values="Content-Length" allowed_sts_header_values="User-Agent" allowed_sts_header_values="X-Amz-Date" allowed_sts_header_values="Authorization" allowed_sts_header_values="X-Amz-Security-Token" allowed_sts_header_values="Host" allowed_sts_header_values="X-Vault-Aws-Iam-Server-Id"

Sorry for the inconvenience and thanks again for highlighting this so quickly.

[fernando-villalba]

@kalafut thank you for responding so quickly as well! I think I will just wait for you guys to patch it and then I'll upgrade to the next version :-)

[fernando-villalba]

By the way, I tried the command you suggested and it was missing one header X-Vault-Aws-Iam-Server-Id

So the command should be for it to work:

vault write auth/aws/config/client allowed_sts_header_values="Content-Type" allowed_sts_header_values="Content-Length" allowed_sts_header_values="User-Agent" allowed_sts_header_values="X-Amz-Date" allowed_sts_header_values="Authorization" allowed_sts_header_values="X-Amz-Security-Token" allowed_sts_header_values="Host" allowed_sts_header_values="X-Vault-Aws-Iam-Server-Id"

Now it seems to be working on 1.5.2 after that.

[kalafut]

@fernando-villalba Thank you. I edited my comment. We'll be publishing this as a Known Issue soon.

[forty]

Is there any special setup that needs to be done before updating allowed_sts_header_values ? The write succeeds but seems to be ignored:

$ vault write /auth/aws/config/client \
>       allowed_sts_header_values="Content-Type" \
>       allowed_sts_header_values="Content-Length" \
>       allowed_sts_header_values="User-Agent" \
>       allowed_sts_header_values="X-Amz-Date" \
>       allowed_sts_header_values="Authorization" \
>       allowed_sts_header_values="X-Amz-Security-Token" \
>       allowed_sts_header_values="Host" \
>       allowed_sts_header_values="X-Vault-Aws-Iam-Server-Id"
Success! Data written to: auth/aws/config/client

$ vault read  /auth/aws/config/client
Key                           Value
---                           -----
access_key                    n/a
endpoint                      n/a
iam_endpoint                  n/a
iam_server_id_header_value    n/a
max_retries                   -1
sts_endpoint                  https://sts.eu-west-1.amazonaws.com

Note: my existing nodes are in 1.2.2, but I tried to failover to a 1.5.2 node, with same results

[sgmiller]

The command line client and server will need to be 1.5.1 or higher for the workaround to take affect.

[forty]

Updating the client seems to have made it work indeed. Is it expected not to see the allowed_sts_header_values when doing vault read /auth/aws/config/client ? (I still don't see them but I do manage to login, so it must have worked)

[sgmiller]

The allowed_sts_header_values indeed is not returned in the read. We're aware this is a bug we will fix along with this issue.

[pbernal]

Closing this issue, as this was addressed in #9825 and is available in 1.5.3. Thank you for reporting this!