Aws-sdk-go: session.New doesn't invoke NewWebIdentityCredentials unless AWS_SDK_LOAD_CONFIG is set

Created on 12 Sep 2019 · 7Comments · Source: aws/aws-sdk-go

Please fill out the sections below to help us address your issue.

Version of AWS SDK for Go?

1.23.19

Version of Go (`go version`)?

1.12.7

What issue did you see?

The title of the issue says it all. See https://github.com/hashicorp/vault/issues/7450#issuecomment-530160521 for more of my investigation. This prevents the SDK from picking up EKS pod service account credentials and exchanging them for AWS credentials when using session.New and AWS_SDK_LOAD_CONFIG isn't set.

session.NewSession doesn't face this limitation. It will always try to invoke NewWebIdentityCredentials regardless of whether AWS_SDK_LOAD_CONFIG is set. It seems weird that there's a divergence in this behavior between the two when the documentation states that session.NewSession is the same as session.New except it the former can return errors.

Steps to reproduce

Haven't tested it out yet, sorry. Mostly just looking through code.

guidance

Source

joelthompson

👍5

Most helpful comment

@diehlaws your output is confirming what @micahhausler and @joelthompson are saying.

With New your assumed role is ... NodeInstanceRole... (the host node's role) and with NewSession it is ...addon-iam-Role1... (the IRSA role).

ameir on 3 Oct 2019

👍3

All 7 comments

Steps to Reproduce

https://play.golang.org/p/9kepGClDdok

```bash

running in an EKS pod

$ env | grep AWS
AWS_ROLE_ARN=arn:aws:iam::111122223333:role/s3-reader
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
$ ls -al $AWS_WEB_IDENTITY_TOKEN_FILE
lrwxrwxrwx 1 root root 12 Sep 6 16:25 /var/run/secrets/eks.amazonaws.com/serviceaccount/token -> ..data/token
$ go run main.go
Creating session via 'NewSession'
{
"Account": "111122223333",
"Arn": "arn:aws:sts::111122223333:assumed-role/s3-reader/1568241943797235661",
"UserId": "XXXX:1568241943797235661"
}
Creating session via 'NewSessionWithOptions'
{
"Account": "111122223333",
"Arn": "arn:aws:sts::111122223333:assumed-role/s3-reader/1568241944243481837",
"UserId": "XXXX:1568241944243481837"
}
Creating session via 'New'
{
"Account": "111122223333",
"Arn": "arn:aws:sts::111122223333:assumed-role/eks-1-14-mhausler-09-04-2019-NODE-NodeInstanceRole-ZKNX5MQZ6D1/i-0614227f41fe18285",
"UserId": "YYYY-0614227f41fe18386"
}

micahhausler on 12 Sep 2019

👍1

Hi @joelthompson, thanks for reaching out to us about this. Unfortunately I haven't been able to reproduce the described behavior, using both session.New() and session.NewSession() on versions 1.23.16 and 1.23.19 results in the expected role being used for the session. Using the code snippet provided by @micahhausler (adding a Println showing the SDK version):

Output

$ kubectl version --short
Client Version: v1.14.6
Server Version: v1.13.10-eks-5ac0f1
$ kubectl exec -it go-2828 bash
bash-4.2# env | grep AWS
AWS_ROLE_ARN=arn:aws:iam::1234567489012:role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
bash-4.2# ls -al $AWS_WEB_IDENTITY_TOKEN_FILE
lrwxrwxrwx 1 root root 12 Sep 17 22:44 /var/run/secrets/eks.amazonaws.com/serviceaccount/token -> ..data/token
bash-4.2# /app/main 
Using AWS SDK for Go version 1.23.16
Creating session via 'New'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa",
  "UserId": "AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa"
}
Creating session via 'NewSession'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568699413159776961",
  "UserId": "AROAIOSFODNN7EXAMPLE:1568699413159776961"
}
Creating session via 'NewSessionWithOptions'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568699413354230079",
  "UserId": "AROAIOSFODNN7EXAMPLE:1568699413354230079"
}


bash-4.2# env | grep AWS
AWS_ROLE_ARN=arn:aws:iam::1234567489012:role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
bash-4.2# ls -al $AWS_WEB_IDENTITY_TOKEN_FILE
lrwxrwxrwx 1 root root 12 Sep 17 22:44 /var/run/secrets/eks.amazonaws.com/serviceaccount/token -> ..data/token
bash-4.2# /app/main 
Using AWS SDK for Go version 1.23.19
Creating session via 'New'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa",
  "UserId": "AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa"
}
Creating session via 'NewSession'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568760345971313203",
  "UserId": "AROAUNIDBI7DY7UA2D75G:1568760345971313203"
}
Creating session via 'NewSessionWithOptions'
{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568760346816309352",
  "UserId": "AROAUNIDBI7DY7UA2D75G:1568760346816309352"
}

Debug Output

Using AWS SDK for Go version 1.23.16
Creating session via 'NewSession'
2019/09/17 17:03:39 DEBUG: Request sts/AssumeRoleWithWebIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Content-Length: 1186
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Encoding: gzip

Action=AssumeRoleWithWebIdentity&RoleArn=arn%3Aaws%3Aiam%3A%3A1234567489012%3Arole%2Feksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0&RoleSessionName=1568739819649562570&Version=2011-06-15&WebIdentityToken=eyJhbGciOiJSUzI1NiIsImtpZCI6ImQwMGY2NjA5ZmMyMGJhYmRhMTliMjc4OTBlMjgzMmZhZWUzNDc4NGUifQ.eyJhdWQiOlsic3RzLmFtYXpvbmF3cy5jb20iXSwiZXhwIjoxNTY4ODI1ODQ4LCJpYXQiOjE1Njg3Mzk0NDgsImlzcyI6Imh0dHBzOi8vb2lkYy5la3MudXMtd2VzdC0yLmFtYXpvbmF3cy5jb20vaWQvQTIxRTI4NUI1OUY2NDQ3QzlFM0UwRDgyMjI5RTEyQ0QiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiIzYjJkZGM5YS1kOTZjLTExZTktYTU3Yi0wMjI4MTJiYjFiYTYifSwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiJiOGRkYmE1NC1kOTBiLTExZTktODEyMy0wYTBmZGFjZGJiZTIifX0sIm5iZiI6MTU2ODczOTQ0OCwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6Z28tMjgyOCJ9.j8mjpBiKbc73xnlIvU9o7N0TZWV_jn8G2hjDXgCxyIxthIUHjnEi2b5CWS-ND8jd7WJX5cXJX_O7634E63vY2GjFN12G7CjD8GZ0W7zs7sTfw6v_96vvWa7ro4GcM4WiB20NFm6TtlIT23TjsbQI_nN19x3B4pP3jeo_jQl56iZUOsfOFpN_zypCrkdoIjgPBZ6cnLvG69jP7yKF9gxkX3wyofBG_WmOucZoq-GvbcnJJS0I4viVs8AurgxQT6bcdRLJ4LD4pt5kurSE6ZkWVNCrHUD6PQWk-enuGyl7uYw84oQkD0JMwMRXJfwgF3lvDY5xp5z0pVUu02sc-AadJg
-----------------------------------------------------
2019/09/17 17:03:39 DEBUG: Response sts/AssumeRoleWithWebIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 1938
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:03:39 GMT
X-Amzn-Requestid: 189fd01f-d96d-11e9-87d8-111dab6cbfdb


-----------------------------------------------------
2019/09/17 17:03:39 <AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739819649562570</Arn>
      <AssumedRoleId>AROAIOSFODNN7EXAMPLE:1568739819649562570</AssumedRoleId>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::1234567489012:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/A21E285B59F6447C9E3E0D82229E12CD</Provider>
    <Credentials>
      <AccessKeyId>ASIAUNIDBI7DQ4PSEJOL</AccessKeyId>
      <SecretAccessKey>SEJ/O95mhVz1io7Ss1iwHLrsFcNRJib2IG30uNc2</SecretAccessKey>
      <SessionToken>FQoGZXIvYXdzEML//////////wEaDI/LUtgzOOyI5lwdlSLFBDywG8UjkgGlxE61t8M9zHl+6LDhM9k5kLPZ8gvmLMkkGeM2uBvwUfFonssv6TLaOwMCEN58jPqkAx2QnIBK1F/kkEbBNnstgqa4FnxnWL+JNhsx3dDDLDAwDL6f/mP63RmZUNUTSfBwFNaaeZObMc9WPrbTxkrFSW0uL5u2WM5rW4k+N7Ea74ElYUVhHNc1mwpXjtqfcyBtHAtEBsm/qig1uweNRVNGWnmnZ+FNUM2YI4k0fd2zuiIaLwJSgDVrS1sg2hQ9Y0GcSEjxOg4bc92yVelAmluixKP1LmYizWiavLnCosJ0YTEcJ7HVIZrLRi+JJJ+WJTa62XLqhbOR7Fst+wf+F3ZiqnMBcGzRo9MelzmwEbg7xN4LjEbV1yBe19N2alOfFeJ0y0cmj/ZMfHNx/bmVNiHdz2DKYa+ZpabZnZHlqlho8uaTkY8Bxil4s6gHDtclW8sOZrMHLlwWNa0iIdnBm2OkYaL6TbzrGFo5l1st+cfPDSYfxOhTBeurQHmJP8lzQdx6y5x4drjrU73WSwwuzY1btoD0tO//2+FjHqW1EvPb7wQp61fhC5yYaabP3gOolfMQ+m67NUuhkOGu6qp14gL6KGPwooJt0jMYINMdrhtaDL3Qk44aXA4hs0s2S9EaeliF7x2TF4GwSbEWelhHD4KYgmBUXaa88IcT+PnBwmT9h93CT/5ju6yG371FXf3wM7ot4YIi/oOCTOWK780lQno6FE8maIq1MeDNALhy9/B6g8xPZkglsStJeb6x5b79KOujhOwF</SessionToken>
      <Expiration>2019-09-17T18:03:39Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:default:go-2828</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>189fd01f-d96d-11e9-87d8-111dab6cbfdb</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

2019/09/17 17:03:39 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7DQ4PSEJOL/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=c1af77b898389ff5720ffcbc84d50f3113bc77161a93dd8fee9a8deda80821cb
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T170339Z
X-Amz-Security-Token: FQoGZXIvYXdzEML//////////wEaDI/LUtgzOOyI5lwdlSLFBDywG8UjkgGlxE61t8M9zHl+6LDhM9k5kLPZ8gvmLMkkGeM2uBvwUfFonssv6TLaOwMCEN58jPqkAx2QnIBK1F/kkEbBNnstgqa4FnxnWL+JNhsx3dDDLDAwDL6f/mP63RmZUNUTSfBwFNaaeZObMc9WPrbTxkrFSW0uL5u2WM5rW4k+N7Ea74ElYUVhHNc1mwpXjtqfcyBtHAtEBsm/qig1uweNRVNGWnmnZ+FNUM2YI4k0fd2zuiIaLwJSgDVrS1sg2hQ9Y0GcSEjxOg4bc92yVelAmluixKP1LmYizWiavLnCosJ0YTEcJ7HVIZrLRi+JJJ+WJTa62XLqhbOR7Fst+wf+F3ZiqnMBcGzRo9MelzmwEbg7xN4LjEbV1yBe19N2alOfFeJ0y0cmj/ZMfHNx/bmVNiHdz2DKYa+ZpabZnZHlqlho8uaTkY8Bxil4s6gHDtclW8sOZrMHLlwWNa0iIdnBm2OkYaL6TbzrGFo5l1st+cfPDSYfxOhTBeurQHmJP8lzQdx6y5x4drjrU73WSwwuzY1btoD0tO//2+FjHqW1EvPb7wQp61fhC5yYaabP3gOolfMQ+m67NUuhkOGu6qp14gL6KGPwooJt0jMYINMdrhtaDL3Qk44aXA4hs0s2S9EaeliF7x2TF4GwSbEWelhHD4KYgmBUXaa88IcT+PnBwmT9h93CT/5ju6yG371FXf3wM7ot4YIi/oOCTOWK780lQno6FE8maIq1MeDNALhy9/B6g8xPZkglsStJeb6x5b79KOujhOwF
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 508
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:03:39 GMT
X-Amzn-Requestid: 18adb2f0-d96d-11e9-87d8-111dab6cbfdb


-----------------------------------------------------
2019/09/17 17:03:40 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739819649562570</Arn>
    <UserId>AROAIOSFODNN7EXAMPLE:1568739819649562570</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>18adb2f0-d96d-11e9-87d8-111dab6cbfdb</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739819649562570",
  "UserId": "AROAIOSFODNN7EXAMPLE:1568739819649562570"
}
Creating session via 'NewSessionWithOptions'
2019/09/17 17:03:40 DEBUG: Request sts/AssumeRoleWithWebIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Content-Length: 1186
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Encoding: gzip

Action=AssumeRoleWithWebIdentity&RoleArn=arn%3Aaws%3Aiam%3A%3A1234567489012%3Arole%2Feksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0&RoleSessionName=1568739820066463126&Version=2011-06-15&WebIdentityToken=eyJhbGciOiJSUzI1NiIsImtpZCI6ImQwMGY2NjA5ZmMyMGJhYmRhMTliMjc4OTBlMjgzMmZhZWUzNDc4NGUifQ.eyJhdWQiOlsic3RzLmFtYXpvbmF3cy5jb20iXSwiZXhwIjoxNTY4ODI1ODQ4LCJpYXQiOjE1Njg3Mzk0NDgsImlzcyI6Imh0dHBzOi8vb2lkYy5la3MudXMtd2VzdC0yLmFtYXpvbmF3cy5jb20vaWQvQTIxRTI4NUI1OUY2NDQ3QzlFM0UwRDgyMjI5RTEyQ0QiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiIzYjJkZGM5YS1kOTZjLTExZTktYTU3Yi0wMjI4MTJiYjFiYTYifSwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiJiOGRkYmE1NC1kOTBiLTExZTktODEyMy0wYTBmZGFjZGJiZTIifX0sIm5iZiI6MTU2ODczOTQ0OCwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6Z28tMjgyOCJ9.j8mjpBiKbc73xnlIvU9o7N0TZWV_jn8G2hjDXgCxyIxthIUHjnEi2b5CWS-ND8jd7WJX5cXJX_O7634E63vY2GjFN12G7CjD8GZ0W7zs7sTfw6v_96vvWa7ro4GcM4WiB20NFm6TtlIT23TjsbQI_nN19x3B4pP3jeo_jQl56iZUOsfOFpN_zypCrkdoIjgPBZ6cnLvG69jP7yKF9gxkX3wyofBG_WmOucZoq-GvbcnJJS0I4viVs8AurgxQT6bcdRLJ4LD4pt5kurSE6ZkWVNCrHUD6PQWk-enuGyl7uYw84oQkD0JMwMRXJfwgF3lvDY5xp5z0pVUu02sc-AadJg
-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response sts/AssumeRoleWithWebIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 1938
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:03:39 GMT
X-Amzn-Requestid: 18b999f1-d96d-11e9-87d8-111dab6cbfdb


-----------------------------------------------------
2019/09/17 17:03:40 <AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739820066463126</Arn>
      <AssumedRoleId>AROAIOSFODNN7EXAMPLE:1568739820066463126</AssumedRoleId>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::1234567489012:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/A21E285B59F6447C9E3E0D82229E12CD</Provider>
    <Credentials>
      <AccessKeyId>ASIAUNIDBI7DXG3SCEES</AccessKeyId>
      <SecretAccessKey>24ewJeSiWZOen1GiVd2K9rTsatVaModV3Lmhs1ia</SecretAccessKey>
      <SessionToken>FQoGZXIvYXdzEML//////////wEaDOj1aY9jQzwWw1L09CLFBDZfthyAaDrjBGlcQaPwV0qTQTNsFSRrBy/dTU5IhS7HwfAVOMau651czfwP/mEjegefFjiOKDzkn6pL/hEABmMXyl3k/f7gnKuilHkswQ8tXD4pw8MbfWRAtYz26+S+PKbgmJDycq1JgD3MEKeMBDJZ27oRWC+qykR8ElASf2OJB1sd3bBytvggMB+h13JdVcdz2ltMfrTwqQqM6Tzb/OtR8HBHxKhk/SNTxMwbjLk/EK3vdY0l/57vuIu/0pQui0Sw6ekbkcfwx2yMZYHK0/Hjtrkfsl7cYWwmlehfjwrZGVZg++oae5FsqYwzlwQRZiIsVqYb1a2zinwpKybhMiIVOULF2fE6TkyTP3r4XR7bcsfNaCrvsQ0+7zVDYgnZqKgok9xssc8VVaah7GvaDZT7w1chGs1ht8UQSkz362HFjMHoApOFDpjgTx2B0SavagOM+0gc84ZBm4Ld5GEIMhFwUM+QBHj9xOWW6AdOaJtzfDwrkaEZAM7xm6+IjFdfJxYe0QEezTn7fXgA6xutvlI+kGmfwHmoSyPitkh6xCUIZ9QKdiHa5Omh9daTiwzw/zk5CWlYoCol5CNB51e24LjLlM/BwIW5r3UJeTADEK6XTfV8NbR1LeGcBSfjpa+gDEL/VJCD45HyCJRbeprKV4erHBqRJHLnLCIZpHQU02q3cYu8J2ip19uSJqZkXcY8gJm/luOhl+1HPsCw5cfbgUB20iOl12yZTNJsxR3blMdZW22hEMR98DTwHoeKuKoMq27BwvrYKOyjhOwF</SessionToken>
      <Expiration>2019-09-17T18:03:40Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:default:go-2828</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>18b999f1-d96d-11e9-87d8-111dab6cbfdb</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

2019/09/17 17:03:40 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7DXG3SCEES/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=131fcea126538b2483cffe41c063c6b2bf507aad232dc65852f834e6f5c51b8c
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T170340Z
X-Amz-Security-Token: FQoGZXIvYXdzEML//////////wEaDOj1aY9jQzwWw1L09CLFBDZfthyAaDrjBGlcQaPwV0qTQTNsFSRrBy/dTU5IhS7HwfAVOMau651czfwP/mEjegefFjiOKDzkn6pL/hEABmMXyl3k/f7gnKuilHkswQ8tXD4pw8MbfWRAtYz26+S+PKbgmJDycq1JgD3MEKeMBDJZ27oRWC+qykR8ElASf2OJB1sd3bBytvggMB+h13JdVcdz2ltMfrTwqQqM6Tzb/OtR8HBHxKhk/SNTxMwbjLk/EK3vdY0l/57vuIu/0pQui0Sw6ekbkcfwx2yMZYHK0/Hjtrkfsl7cYWwmlehfjwrZGVZg++oae5FsqYwzlwQRZiIsVqYb1a2zinwpKybhMiIVOULF2fE6TkyTP3r4XR7bcsfNaCrvsQ0+7zVDYgnZqKgok9xssc8VVaah7GvaDZT7w1chGs1ht8UQSkz362HFjMHoApOFDpjgTx2B0SavagOM+0gc84ZBm4Ld5GEIMhFwUM+QBHj9xOWW6AdOaJtzfDwrkaEZAM7xm6+IjFdfJxYe0QEezTn7fXgA6xutvlI+kGmfwHmoSyPitkh6xCUIZ9QKdiHa5Omh9daTiwzw/zk5CWlYoCol5CNB51e24LjLlM/BwIW5r3UJeTADEK6XTfV8NbR1LeGcBSfjpa+gDEL/VJCD45HyCJRbeprKV4erHBqRJHLnLCIZpHQU02q3cYu8J2ip19uSJqZkXcY8gJm/luOhl+1HPsCw5cfbgUB20iOl12yZTNJsxR3blMdZW22hEMR98DTwHoeKuKoMq27BwvrYKOyjhOwF
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 508
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:03:39 GMT
X-Amzn-Requestid: 18c5a7fe-d96d-11e9-87d8-111dab6cbfdb


-----------------------------------------------------
2019/09/17 17:03:40 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739820066463126</Arn>
    <UserId>AROAIOSFODNN7EXAMPLE:1568739820066463126</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>18c5a7fe-d96d-11e9-87d8-111dab6cbfdb</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568739820066463126",
  "UserId": "AROAIOSFODNN7EXAMPLE:1568739820066463126"
}
Creating session via 'New'
2019/09/17 17:03:40 DEBUG: Request ec2metadata/GetMetadata Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /latest/meta-data/iam/security-credentials/ HTTP/1.1
Host: 169.254.169.254
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Accept-Encoding: gzip


-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response ec2metadata/GetMetadata Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 64
Accept-Ranges: none
Content-Type: text/plain
Date: Tue, 17 Sep 2019 17:03:40 GMT
Last-Modified: Tue, 17 Sep 2019 16:57:36 GMT
Server: EC2ws


-----------------------------------------------------
2019/09/17 17:03:40 eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL
2019/09/17 17:03:40 DEBUG: Request ec2metadata/GetMetadata Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /latest/meta-data/iam/security-credentials/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL HTTP/1.1
Host: 169.254.169.254
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Accept-Encoding: gzip


-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response ec2metadata/GetMetadata Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 1286
Accept-Ranges: none
Content-Type: text/plain
Date: Tue, 17 Sep 2019 17:03:40 GMT
Last-Modified: Tue, 17 Sep 2019 16:57:36 GMT
Server: EC2ws


-----------------------------------------------------
2019/09/17 17:03:40 {
  "Code" : "Success",
  "LastUpdated" : "2019-09-17T16:09:17Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAUNIDBI7D45EPWHFE",
  "SecretAccessKey" : "NnKBSlYYPQgea9OWWkbgIKEYkzgZe8wzKz9ELiQ5",
  "Token" : "AgoJb3JpZ2luX2VjELD//////////wEaCXVzLXdlc3QtMiJGMEQCIGa2n4eGpZ5d04rOYIjIJ8dFsdfINf03aIOWFg51B3X7AiBwV0fVu37QYf2yMUm98l8FE/dqn2gxoxRoA96tq/Zt/SraAwh5EAEaDDMwMzMzODQ0MDY0NyIMzbNOt4DYGqomza9bKrcDWLW3SIchHutqPdMIuRDR3dTs+w13K9S8mUer2R5HpT4pZWJYPpcLLfFFElurD1VT4WWH1a55Xr68Oz0KQQwnbWg3DChROj4Ob073Z01RiB7iGqtakJ6lvANaRekJkGtxJTE8X3JN4GiZ70/TE5RZ8Rw4zubHtOyXR8KflCLkVFm+94n9s5zZlXX4rgaIB/Xy4G17SddOjQEfmMfxmaAccY+JcHbFDfakYi+X+Qh6gMQaaAch6pA1XRe8GytxO57FGFJLmL0Jidnwx3lzEZLnxq0+I/q/yCSnPgZHgd5oy6o9nfcJa6px5PcR6WlQqqU+KBCHAx9IKc6BfH/Q01ph65snV3vWQyYajGyh/X2nVUYjhmuabY6MRBKH6WR6MtZf3p3ln0TNsYp5cp6GHQayQ6HKH/VoW+WqPL5BaVEEgt5Qjo4NCFM52zJPOQBixd5EKBHzbL/+ueOCBzcwLmUTqREE/Ex1BfYNVyKWIXHHWnSMUupZSLyQ6cixCu0xH2SNFjvUiMPHfji3erDNdlknWXhKlQqgRgN0YR+H5TUZKvQ1rjve3CPHlgtqueZwHH1ME9s0QTkGujCVioTsBTq1AbB6nniRwAt+exlnY+6TrOYoJGx5YIqUHAE5vkaPDWJ2VzUInJCT1nuiRr58cVPXCN/DCkgwr/rUY7K4E20eBuBqRAzLcUWzw0/jKUILjAxCu1U30WD8cQYzuxeg2VkKq1sbS7vaBNBZlxouChDca3mD67k2iUM0PFf5BkKVnHbuzQyBNs9EsKQpvFxZQsIUT6SwohWt7spPaXMz9p3PXSMZtytPkuGKjJMf9e93BSVNGy0D/Ms=",
  "Expiration" : "2019-09-17T22:41:57Z"
}
2019/09/17 17:03:40 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.16 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7D45EPWHFE/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=dca263a9743dcca7f7cccb644f78bdd27045c4cab21d601783bf6733e1142e27
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T170340Z
X-Amz-Security-Token: AgoJb3JpZ2luX2VjELD//////////wEaCXVzLXdlc3QtMiJGMEQCIGa2n4eGpZ5d04rOYIjIJ8dFsdfINf03aIOWFg51B3X7AiBwV0fVu37QYf2yMUm98l8FE/dqn2gxoxRoA96tq/Zt/SraAwh5EAEaDDMwMzMzODQ0MDY0NyIMzbNOt4DYGqomza9bKrcDWLW3SIchHutqPdMIuRDR3dTs+w13K9S8mUer2R5HpT4pZWJYPpcLLfFFElurD1VT4WWH1a55Xr68Oz0KQQwnbWg3DChROj4Ob073Z01RiB7iGqtakJ6lvANaRekJkGtxJTE8X3JN4GiZ70/TE5RZ8Rw4zubHtOyXR8KflCLkVFm+94n9s5zZlXX4rgaIB/Xy4G17SddOjQEfmMfxmaAccY+JcHbFDfakYi+X+Qh6gMQaaAch6pA1XRe8GytxO57FGFJLmL0Jidnwx3lzEZLnxq0+I/q/yCSnPgZHgd5oy6o9nfcJa6px5PcR6WlQqqU+KBCHAx9IKc6BfH/Q01ph65snV3vWQyYajGyh/X2nVUYjhmuabY6MRBKH6WR6MtZf3p3ln0TNsYp5cp6GHQayQ6HKH/VoW+WqPL5BaVEEgt5Qjo4NCFM52zJPOQBixd5EKBHzbL/+ueOCBzcwLmUTqREE/Ex1BfYNVyKWIXHHWnSMUupZSLyQ6cixCu0xH2SNFjvUiMPHfji3erDNdlknWXhKlQqgRgN0YR+H5TUZKvQ1rjve3CPHlgtqueZwHH1ME9s0QTkGujCVioTsBTq1AbB6nniRwAt+exlnY+6TrOYoJGx5YIqUHAE5vkaPDWJ2VzUInJCT1nuiRr58cVPXCN/DCkgwr/rUY7K4E20eBuBqRAzLcUWzw0/jKUILjAxCu1U30WD8cQYzuxeg2VkKq1sbS7vaBNBZlxouChDca3mD67k2iUM0PFf5BkKVnHbuzQyBNs9EsKQpvFxZQsIUT6SwohWt7spPaXMz9p3PXSMZtytPkuGKjJMf9e93BSVNGy0D/Ms=
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:03:40 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 509
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:03:39 GMT
X-Amzn-Requestid: 18d22b35-d96d-11e9-87d8-111dab6cbfdb


-----------------------------------------------------
2019/09/17 17:03:40 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa</Arn>
    <UserId>AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>18d22b35-d96d-11e9-87d8-111dab6cbfdb</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa",
  "UserId": "AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa"
}

---------------------------------------

Using AWS SDK for Go version 1.23.22
Creating session via 'New'
2019/09/17 17:54:26 DEBUG: Request ec2metadata/GetMetadata Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /latest/meta-data/iam/security-credentials/ HTTP/1.1
Host: 169.254.169.254
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Accept-Encoding: gzip


-----------------------------------------------------
2019/09/17 17:54:26 DEBUG: Response ec2metadata/GetMetadata Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 64
Accept-Ranges: none
Content-Type: text/plain
Date: Tue, 17 Sep 2019 17:54:26 GMT
Last-Modified: Tue, 17 Sep 2019 16:57:36 GMT
Server: EC2ws


-----------------------------------------------------
2019/09/17 17:54:26 eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL
2019/09/17 17:54:26 DEBUG: Request ec2metadata/GetMetadata Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /latest/meta-data/iam/security-credentials/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL HTTP/1.1
Host: 169.254.169.254
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Accept-Encoding: gzip


-----------------------------------------------------
2019/09/17 17:54:26 DEBUG: Response ec2metadata/GetMetadata Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 1286
Accept-Ranges: none
Content-Type: text/plain
Date: Tue, 17 Sep 2019 17:54:26 GMT
Last-Modified: Tue, 17 Sep 2019 17:08:55 GMT
Server: EC2ws


-----------------------------------------------------
2019/09/17 17:54:26 {
  "Code" : "Success",
  "LastUpdated" : "2019-09-17T17:09:13Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAUNIDBI7DQG5FTJKP",
  "SecretAccessKey" : "ozqE1FmQyRtTc2MCgYlPgYvs7vgmVahR6BRrbgtQ",
  "Token" : "AgoJb3JpZ2luX2VjELH//////////wEaCXVzLXdlc3QtMiJIMEYCIQDMsvhipXslA3lnLz0Oxvx/wEICBRhPaRLB2bJg2Zkb0QIhAPN16VlK4OXVkQIdK4M3j7Hbx0LP/MF00Q15wB+0wd/2KtoDCHoQARoMMzAzMzM4NDQwNjQ3IgxnRBVA7EY2f6PaZ9AqtwNd48bHtHg2+ypyttB7gsYgiu8oGtckYPiLp+ejydRruvxZUW6+H8WZ6wIv+68BbOyQwKve6osIo7AUlBKdPUgiCJoi6A+UcyOlHVqxZQ2ZhubdoIf40VxrCzN3FlF57zkmEjh4c2josdjQOiXNwV46aUzf/7DTsoKe7d5Kp4yVQJ68RgTrvwr0cXnELdbcNHFwDXD5xv1SQJnJX6ThSINBDRSMlNxsf94zArnhe3FOWMPPT+Fk6ik0pXzR/Qh7F759l323UhDo63gMYpcBh9rnrxl/UWCE63MwpH/rkFgoQkDYJWrttW5EfCkkRaZ9MPyQ/HehOnKBVqt2kYYPVfdf5I+C31pmFGvDsRBrj1/ZeJVnIGWlpqRCWo7ApXg5327ge9WP3nKSAKfeAxANdzFmDZ20tk1/6nmOSX3XYVkUhz+z/cY+r3TsTXypVBmk3eCAk87T8JxAKUsUXZ9QKlqrxFqY6PGwY3vZlPkbjvlgx63t9D7vf3BjjYqyVyeIbG2Bamu+e6+1pzte9uibHTcpqOIIJvLzTbrnGpLfqsxTYRZh/X2axanABzQHDuFBTikAQz1BuHZLMKWmhOwFOrMBVpxhsCUCHe8llh7MbYW4JISbYXv4Y4Ksoo0yWELUM3mrcw5jRf8CCdKrb6AFWGc2UMhwLmhdmN6LmJD6ZTKv4au4SwuH6+hgbGMx/mFJom0cEZ977m3ZdSISXno9tHPQtXmu8ojRB3levpMej2FIMV1bRk07CVVDPzeUi0cZdA2QpzEPWMazlQSkUPGaNw1xYOVRlnM7mDjG0FEC2MswXMB/t4RVptmH5x1+2oaZnemWTNw=",
  "Expiration" : "2019-09-17T23:27:44Z"
}
2019/09/17 17:54:26 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7DQG5FTJKP/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=5f6bdb558feb1b4981b356e25501ce95db155be0a0a0d00d05a551808448abc5
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T175426Z
X-Amz-Security-Token: AgoJb3JpZ2luX2VjELH//////////wEaCXVzLXdlc3QtMiJIMEYCIQDMsvhipXslA3lnLz0Oxvx/wEICBRhPaRLB2bJg2Zkb0QIhAPN16VlK4OXVkQIdK4M3j7Hbx0LP/MF00Q15wB+0wd/2KtoDCHoQARoMMzAzMzM4NDQwNjQ3IgxnRBVA7EY2f6PaZ9AqtwNd48bHtHg2+ypyttB7gsYgiu8oGtckYPiLp+ejydRruvxZUW6+H8WZ6wIv+68BbOyQwKve6osIo7AUlBKdPUgiCJoi6A+UcyOlHVqxZQ2ZhubdoIf40VxrCzN3FlF57zkmEjh4c2josdjQOiXNwV46aUzf/7DTsoKe7d5Kp4yVQJ68RgTrvwr0cXnELdbcNHFwDXD5xv1SQJnJX6ThSINBDRSMlNxsf94zArnhe3FOWMPPT+Fk6ik0pXzR/Qh7F759l323UhDo63gMYpcBh9rnrxl/UWCE63MwpH/rkFgoQkDYJWrttW5EfCkkRaZ9MPyQ/HehOnKBVqt2kYYPVfdf5I+C31pmFGvDsRBrj1/ZeJVnIGWlpqRCWo7ApXg5327ge9WP3nKSAKfeAxANdzFmDZ20tk1/6nmOSX3XYVkUhz+z/cY+r3TsTXypVBmk3eCAk87T8JxAKUsUXZ9QKlqrxFqY6PGwY3vZlPkbjvlgx63t9D7vf3BjjYqyVyeIbG2Bamu+e6+1pzte9uibHTcpqOIIJvLzTbrnGpLfqsxTYRZh/X2axanABzQHDuFBTikAQz1BuHZLMKWmhOwFOrMBVpxhsCUCHe8llh7MbYW4JISbYXv4Y4Ksoo0yWELUM3mrcw5jRf8CCdKrb6AFWGc2UMhwLmhdmN6LmJD6ZTKv4au4SwuH6+hgbGMx/mFJom0cEZ977m3ZdSISXno9tHPQtXmu8ojRB3levpMej2FIMV1bRk07CVVDPzeUi0cZdA2QpzEPWMazlQSkUPGaNw1xYOVRlnM7mDjG0FEC2MswXMB/t4RVptmH5x1+2oaZnemWTNw=
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:54:26 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 509
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:54:26 GMT
X-Amzn-Requestid: 30c63392-d974-11e9-a7d4-07a615b3af34


-----------------------------------------------------
2019/09/17 17:54:26 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa</Arn>
    <UserId>AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>30c63392-d974-11e9-a7d4-07a615b3af34</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-156869523-NodeInstanceRole-1CXG17IBWMRHL/i-0a6c5f3d09a284efa",
  "UserId": "AROAUNIDBI7D74SNFE3MC:i-0a6c5f3d09a284efa"
}
Creating session via 'NewSession'
2019/09/17 17:54:26 DEBUG: Request sts/AssumeRoleWithWebIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Content-Length: 1186
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Encoding: gzip

Action=AssumeRoleWithWebIdentity&RoleArn=arn%3Aaws%3Aiam%3A%3A1234567489012%3Arole%2Feksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0&RoleSessionName=1234567890123456789&Version=2011-06-15&WebIdentityToken=eyJhbGciOiJSUzI1NiIsImtpZCI6ImQwMGY2NjA5ZmMyMGJhYmRhMTliMjc4OTBlMjgzMmZhZWUzNDc4NGUifQ.eyJhdWQiOlsic3RzLmFtYXpvbmF3cy5jb20iXSwiZXhwIjoxNTY4ODI1ODQ4LCJpYXQiOjE1Njg3Mzk0NDgsImlzcyI6Imh0dHBzOi8vb2lkYy5la3MudXMtd2VzdC0yLmFtYXpvbmF3cy5jb20vaWQvQTIxRTI4NUI1OUY2NDQ3QzlFM0UwRDgyMjI5RTEyQ0QiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiIzYjJkZGM5YS1kOTZjLTExZTktYTU3Yi0wMjI4MTJiYjFiYTYifSwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiJiOGRkYmE1NC1kOTBiLTExZTktODEyMy0wYTBmZGFjZGJiZTIifX0sIm5iZiI6MTU2ODczOTQ0OCwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6Z28tMjgyOCJ9.j8mjpBiKbc73xnlIvU9o7N0TZWV_jn8G2hjDXgCxyIxthIUHjnEi2b5CWS-ND8jd7WJX5cXJX_O7634E63vY2GjFN12G7CjD8GZ0W7zs7sTfw6v_96vvWa7ro4GcM4WiB20NFm6TtlIT23TjsbQI_nN19x3B4pP3jeo_jQl56iZUOsfOFpN_zypCrkdoIjgPBZ6cnLvG69jP7yKF9gxkX3wyofBG_WmOucZoq-GvbcnJJS0I4viVs8AurgxQT6bcdRLJ4LD4pt5kurSE6ZkWVNCrHUD6PQWk-enuGyl7uYw84oQkD0JMwMRXJfwgF3lvDY5xp5z0pVUu02sc-AadJg
-----------------------------------------------------
2019/09/17 17:54:27 DEBUG: Response sts/AssumeRoleWithWebIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 1938
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:54:27 GMT
X-Amzn-Requestid: 30d3522c-d974-11e9-a7d4-07a615b3af34


-----------------------------------------------------
2019/09/17 17:54:27 <AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1234567890123456789</Arn>
      <AssumedRoleId>AROAIOSFODNN7EXAMPLE:1234567890123456789</AssumedRoleId>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::1234567489012:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/A21E285B59F6447C9E3E0D82229E12CD</Provider>
    <Credentials>
      <AccessKeyId>ASIAUNIDBI7D35VY3G4C</AccessKeyId>
      <SecretAccessKey>u+0ig+tsM+Pu8M69kv9YvlI1sfJbGYLs1CpTjjIR</SecretAccessKey>
      <SessionToken>FQoGZXIvYXdzEMP//////////wEaDOncZBYeH775JmNvDyLFBGETpNVxL2Nn0lRY3Cva6B/BvyguXVAbrkg8dLR/fotBMOMCx0MLM3m8Tx+hmVxNmCk+Gxo3SD8p4duUnyd5ouYpdg8qTJk2B+kizUXrSTA5chaWS22KUrpneBl1pbu53B11RjWIWAskaBWtrj+Vpmm/y0i64xSZvcehSMoYf/UNkRbYp8DoQHUcO0fOourUUaYKCq/8yuUoikfnWTV/5yGzNuZ1zy1igep8biCCI5PHyFSPzxhnvjkZTP8S+uOyRE3/WpTyiCowXwIrRmqfMyfyMXZSxaEgZvOXmXJI0lvaP/kZgP7pZfmAmN5lvvdEveaW/Vy+cJNqfZhyGqXzsw4tC/FbaDayh6KtZf7A2CO5PUIiUCGaFIdTlKbOssKr8YeMeBA1+gQZE2UnmJOMez9inxd7eYoH+a3Ei8V51GYQZUDkrczkXKOzfA6XTdlHEjiYfXfD8P67LcSclqznN7SjvGymDICJEHqcSKIJPk8YzW3B95Sv3OUK9jqLwHigPvkwT3XjcYMIVHqjHM/FcX4RqSpEC/pkXjPd1Uj3DzRTJ4cPPRmn5Cs3wef4OKpWK9SQILIU5SyvwbGCNjC4X/RFhMwjnZ5Yi2dZIKMTLlxTgS0id4vT9FJxgwKyJXiwoFLHGl/mZLU5mOkYp8Yh0koX0DWYkS/1OHi+QRHuIGN3tRawuSWIchsgLK2TP2MWUmbHh9SI0PUxeefzEl6VVtGeT9FBKFkPvcKwnJaC01pznxP3PL/UknbCGGHYfqkGt/LjU3yYKNO7hOwF</SessionToken>
      <Expiration>2019-09-17T18:54:27Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:default:go-2828</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>30d3522c-d974-11e9-a7d4-07a615b3af34</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

2019/09/17 17:54:27 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7D35VY3G4C/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=4ccdde1f2b5744e3698b8461c4a1ab0da1ff8381c3c439ea89b5e903181da65d
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T175426Z
X-Amz-Security-Token: FQoGZXIvYXdzEMP//////////wEaDOncZBYeH775JmNvDyLFBGETpNVxL2Nn0lRY3Cva6B/BvyguXVAbrkg8dLR/fotBMOMCx0MLM3m8Tx+hmVxNmCk+Gxo3SD8p4duUnyd5ouYpdg8qTJk2B+kizUXrSTA5chaWS22KUrpneBl1pbu53B11RjWIWAskaBWtrj+Vpmm/y0i64xSZvcehSMoYf/UNkRbYp8DoQHUcO0fOourUUaYKCq/8yuUoikfnWTV/5yGzNuZ1zy1igep8biCCI5PHyFSPzxhnvjkZTP8S+uOyRE3/WpTyiCowXwIrRmqfMyfyMXZSxaEgZvOXmXJI0lvaP/kZgP7pZfmAmN5lvvdEveaW/Vy+cJNqfZhyGqXzsw4tC/FbaDayh6KtZf7A2CO5PUIiUCGaFIdTlKbOssKr8YeMeBA1+gQZE2UnmJOMez9inxd7eYoH+a3Ei8V51GYQZUDkrczkXKOzfA6XTdlHEjiYfXfD8P67LcSclqznN7SjvGymDICJEHqcSKIJPk8YzW3B95Sv3OUK9jqLwHigPvkwT3XjcYMIVHqjHM/FcX4RqSpEC/pkXjPd1Uj3DzRTJ4cPPRmn5Cs3wef4OKpWK9SQILIU5SyvwbGCNjC4X/RFhMwjnZ5Yi2dZIKMTLlxTgS0id4vT9FJxgwKyJXiwoFLHGl/mZLU5mOkYp8Yh0koX0DWYkS/1OHi+QRHuIGN3tRawuSWIchsgLK2TP2MWUmbHh9SI0PUxeefzEl6VVtGeT9FBKFkPvcKwnJaC01pznxP3PL/UknbCGGHYfqkGt/LjU3yYKNO7hOwF
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:54:27 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 508
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:54:27 GMT
X-Amzn-Requestid: 314ca1ba-d974-11e9-a7d4-07a615b3af34


-----------------------------------------------------
2019/09/17 17:54:27 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1234567890123456789</Arn>
    <UserId>AROAIOSFODNN7EXAMPLE:1234567890123456789</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>314ca1ba-d974-11e9-a7d4-07a615b3af34</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1234567890123456789",
  "UserId": "AROAIOSFODNN7EXAMPLE:1234567890123456789"
}
Creating session via 'NewSessionWithOptions'
2019/09/17 17:54:27 DEBUG: Request sts/AssumeRoleWithWebIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Content-Length: 1186
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Encoding: gzip

Action=AssumeRoleWithWebIdentity&RoleArn=arn%3Aaws%3Aiam%3A%3A1234567489012%3Arole%2Feksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0&RoleSessionName=1568742867853349086&Version=2011-06-15&WebIdentityToken=eyJhbGciOiJSUzI1NiIsImtpZCI6ImQwMGY2NjA5ZmMyMGJhYmRhMTliMjc4OTBlMjgzMmZhZWUzNDc4NGUifQ.eyJhdWQiOlsic3RzLmFtYXpvbmF3cy5jb20iXSwiZXhwIjoxNTY4ODI1ODQ4LCJpYXQiOjE1Njg3Mzk0NDgsImlzcyI6Imh0dHBzOi8vb2lkYy5la3MudXMtd2VzdC0yLmFtYXpvbmF3cy5jb20vaWQvQTIxRTI4NUI1OUY2NDQ3QzlFM0UwRDgyMjI5RTEyQ0QiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiIzYjJkZGM5YS1kOTZjLTExZTktYTU3Yi0wMjI4MTJiYjFiYTYifSwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImdvLTI4MjgiLCJ1aWQiOiJiOGRkYmE1NC1kOTBiLTExZTktODEyMy0wYTBmZGFjZGJiZTIifX0sIm5iZiI6MTU2ODczOTQ0OCwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6Z28tMjgyOCJ9.j8mjpBiKbc73xnlIvU9o7N0TZWV_jn8G2hjDXgCxyIxthIUHjnEi2b5CWS-ND8jd7WJX5cXJX_O7634E63vY2GjFN12G7CjD8GZ0W7zs7sTfw6v_96vvWa7ro4GcM4WiB20NFm6TtlIT23TjsbQI_nN19x3B4pP3jeo_jQl56iZUOsfOFpN_zypCrkdoIjgPBZ6cnLvG69jP7yKF9gxkX3wyofBG_WmOucZoq-GvbcnJJS0I4viVs8AurgxQT6bcdRLJ4LD4pt5kurSE6ZkWVNCrHUD6PQWk-enuGyl7uYw84oQkD0JMwMRXJfwgF3lvDY5xp5z0pVUu02sc-AadJg
-----------------------------------------------------
2019/09/17 17:54:27 DEBUG: Response sts/AssumeRoleWithWebIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 1938
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:54:27 GMT
X-Amzn-Requestid: 31594b2b-d974-11e9-a7d4-07a615b3af34


-----------------------------------------------------
2019/09/17 17:54:27 <AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568742867853349086</Arn>
      <AssumedRoleId>AROAIOSFODNN7EXAMPLE:1568742867853349086</AssumedRoleId>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::1234567489012:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/A21E285B59F6447C9E3E0D82229E12CD</Provider>
    <Credentials>
      <AccessKeyId>ASIAUNIDBI7D6A2NT3GT</AccessKeyId>
      <SecretAccessKey>Uh6kW4d8QstFmUAHbWjX4ECeI7NR4tu4eL754hOl</SecretAccessKey>
      <SessionToken>FQoGZXIvYXdzEMP//////////wEaDBZmKz8+C64pigi8qCLFBFOUr5IbNj0DVJfxkJNhB5l0sSSAIpJ+ID84QD5bZqEPd7uUf+CYGWWFdLCTo9QhXgqjOhqltGrfYjGrGzwjeBXI2Az4SAhGEQixO2lDiHgzOXaYeC8+T4SOkrqzNuTxQ1b/tiXdEk2t7RWL8lT3fxgVTHhqF5pvAHPw9sGVB+miS2t24HDNZviAngYjwLGV3FVkZ83Zmsb9H8rmHcYbDtTL9rQooXy+XaGGamdkwYYaIxq/xEGjXQwILQopvHBQzFHJuCXx8fI03VsPk2c2lF/a7lufJ4HBX4LfsS6uscswXlOBFYijahuGfj4nIWwHuZwT7RrWN55hv79DQ7AcvpwysI0Z2GMfQik4OpZn7qLyTKXdwqWbS9Lu6ACAHg5eExEGHicFsq2oR03XTNH1bjAYkHTSQlK1koLLASJ+QGUFDhCjLWHAIiC/DpZ7lOdOIe2uUi2YeZHbKg/lVUFGbnfiiuNlD0YVO8LdYysdVk2/dYUlQN9eVAiovFhz2Zfi6dqXV5azT6lHkQmtl4T7HCDOLVNY/CVlTP9AGLPpWKfs3F9lrmPmeSOImgpvOlLIJR+QVO+MoPaZReniHibYLWjFi/vBTyRLxpFFDw0EjpdOrxSkzmZP8deqePi28eDxEeDc79aX8nJDAH5sbcKXslkFqkPR3lMT8bSSjzvocj6X7PToTcVed94kIlHFSSCr0IEPifiFKB3qFTL9dtvhw+pq+7AkISWPHa+ivSyJJQEzWdIkrkCsrzLlzR7709Y6Y2ePhU12KNO7hOwF</SessionToken>
      <Expiration>2019-09-17T18:54:27Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:default:go-2828</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>31594b2b-d974-11e9-a7d4-07a615b3af34</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

2019/09/17 17:54:27 DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.23.22 (go1.12.7; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=ASIAUNIDBI7D6A2NT3GT/20190917/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=c63eb16846af40b6a8b3bd3feb448fc3b5d4f8422c1467828ccba0aae025ceb4
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20190917T175427Z
X-Amz-Security-Token: FQoGZXIvYXdzEMP//////////wEaDBZmKz8+C64pigi8qCLFBFOUr5IbNj0DVJfxkJNhB5l0sSSAIpJ+ID84QD5bZqEPd7uUf+CYGWWFdLCTo9QhXgqjOhqltGrfYjGrGzwjeBXI2Az4SAhGEQixO2lDiHgzOXaYeC8+T4SOkrqzNuTxQ1b/tiXdEk2t7RWL8lT3fxgVTHhqF5pvAHPw9sGVB+miS2t24HDNZviAngYjwLGV3FVkZ83Zmsb9H8rmHcYbDtTL9rQooXy+XaGGamdkwYYaIxq/xEGjXQwILQopvHBQzFHJuCXx8fI03VsPk2c2lF/a7lufJ4HBX4LfsS6uscswXlOBFYijahuGfj4nIWwHuZwT7RrWN55hv79DQ7AcvpwysI0Z2GMfQik4OpZn7qLyTKXdwqWbS9Lu6ACAHg5eExEGHicFsq2oR03XTNH1bjAYkHTSQlK1koLLASJ+QGUFDhCjLWHAIiC/DpZ7lOdOIe2uUi2YeZHbKg/lVUFGbnfiiuNlD0YVO8LdYysdVk2/dYUlQN9eVAiovFhz2Zfi6dqXV5azT6lHkQmtl4T7HCDOLVNY/CVlTP9AGLPpWKfs3F9lrmPmeSOImgpvOlLIJR+QVO+MoPaZReniHibYLWjFi/vBTyRLxpFFDw0EjpdOrxSkzmZP8deqePi28eDxEeDc79aX8nJDAH5sbcKXslkFqkPR3lMT8bSSjzvocj6X7PToTcVed94kIlHFSSCr0IEPifiFKB3qFTL9dtvhw+pq+7AkISWPHa+ivSyJJQEzWdIkrkCsrzLlzR7709Y6Y2ePhU12KNO7hOwF
Accept-Encoding: gzip

Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2019/09/17 17:54:28 DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 508
Content-Type: text/xml
Date: Tue, 17 Sep 2019 17:54:27 GMT
X-Amzn-Requestid: 31666ac9-d974-11e9-a7d4-07a615b3af34


-----------------------------------------------------
2019/09/17 17:54:28 <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568742867853349086</Arn>
    <UserId>AROAIOSFODNN7EXAMPLE:1568742867853349086</UserId>
    <Account>1234567489012</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>31666ac9-d974-11e9-a7d4-07a615b3af34</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>

{
  "Account": "1234567489012",
  "Arn": "arn:aws:sts::1234567489012:assumed-role/eksctl-ferocious-gopher-1568695237-addon-iam-Role1-1YEIGTA6G3B0/1568742867853349086",
  "UserId": "AROAIOSFODNN7EXAMPLE:1568742867853349086"
}

That being said, session.New() is deprecated at this time, so we recommend using session.NewSession() instead to initialize the session to be used by your service client. Another thing to note is that only the shared config file requires the AWS_SDK_LOAD_CONFIG environment variable to retrieve configuration options - are you using this file to specify any config options that should be loaded in the new session using the web identity credentials?

diehlaws on 20 Sep 2019

@diehlaws your output is confirming what @micahhausler and @joelthompson are saying.

With New your assumed role is ... NodeInstanceRole... (the host node's role) and with NewSession it is ...addon-iam-Role1... (the IRSA role).

ameir on 3 Oct 2019

👍3

The problem seems to be that the AWS_WEB_IDENTITY_TOKEN_FILE environment variable is evaluated only in resolveCredentials, which in turn is only invoked in mergeConfigSrcs, which in turn is only invoked in newSession, which is only invoked in NewSession(WithOptions), but not in New.

@diehlaws @micahhausler WIll this be fixed considering that the New function is deprecated?

devkid on 7 Oct 2019

I'm hitting the same issue with github.com/aws/[email protected] on our EKS cluster.

The session.New() doesn't assume the correct role from AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE env vars issued by ServiceAccount object automatically.