Eksctl: Unable to create cluster using profile

[richardcase]

I'm trying to create a cluster with using a specific profile. I've set the AWS_PROFILE environment variable (as per this) but its still creating the cluster using the default cli profile.

We use specific profiles to distinguish between our environments (dev, non-prod, prod) and we're also using MFA.

Looking at #56 it would appear that using profiles can work?

[errordeveloper]

I use AWS_PROFILE in conjunction with MFA all the time when I work on the code, it's ought to work. Also, please keep in mind that the environment variables are picked up by the official AWS library, we have no code to handle this and I would be very surprised if we managed to break it somehow.

There is now #56 to print profile in use, which should easy enough to implement. However, for now, if you want a non-disruptive way to determine the right profile is used, you can use eksctl get clusters --verbose=4 and you should see your role ARN printed at the top, it will include account ID we you should be able to match to your profile.

[errordeveloper]

@richardcase did you manage to create a cluster or not yet?

[richardcase]

@errordeveloper - thanks, i haven't had time to try but will try today and let you know. I'm sure its a case of PEBKAC.

[errordeveloper]

@richardcase see https://github.com/weaveworks/eksctl/issues/56#issuecomment-396064846, and please let us know if that would be helpful.

[richardcase]

Yes that would be useful.

I thought i'd check the AWS cli with the AWS_PROFILE environment variable set and it correctly uses the profile and i get the output from the correct role/account.

But if i then use eksctl with the same environment variables set i now get errors:

keksctl\2018-06-11T08:01:38+01:00 [✖]  checking AWS STS access – cannot get role ARN for current session: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, XXX-dev.
EC2RoleRequestError: no EC2 instance role found
caused by: RequestError: send request failed
caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials: dial tcp 169.254.169.254:80: connect: host is down

I'm guessing that the AWS cli does something extra when running or some subtle differneces between the Python SDK and Go SDK.

I will keep on looking,

[richardcase]

@errordeveloper - i managed to get a cluster created. I had to use the following first:

aws sts assume-role ....

And then use the output to set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN. After which i could create the cluster using eksctl create cluster.

If i use the aws cli i only need to set AWS_PROFILE (or --profile) and not the other environment variables. I guess the CLI is doing something extra (doing the assume role). Is this something eksctl could also do?

[vkhatri]

I had to create a default profile (~/.aws/credentials) to get eksctl working.

Having an option --profile would be nice too along with AWS_PROFILE support.

[richardcase]

I have default profile and eksctl was fine with that. Its just when i wanted to use a different profile (which is a different account and role).

I used a variation of this script or awsudo to make it easier to set the required variables.

But --profile or AWS_PROFILE would be easier.

[richardcase]

Having had a look, the aws cli uses botocore which looks at creating credentails in a number of ways. Have a look at create_credential_resolver in this.

It should be fairly straight forward to add something similar to eksctl as there is the ChainProvider in the Go SDK which could then be used to chain the shared credentials, environment variables and assume role providers (looks like the first 2 are the defaults).

If you want i'd be happy to have a look at this?

[rade]

@richardcase

If you want i'd be happy to have a look at this?

That would be awesome. Quite a few people are running into this problem.

[richardcase]

@rade - i'll have a look at adding this tomorrow and will make a PR when its done.

[richardcase]

If using MFA this will ask for a code everytime its run and doesn't cache them between runs.

[marccarre]

@richardcase, would it be possible to get more details on your initial setup (e.g. eksctl version, local AWS profile/config files, env. vars., configuration/profiles in AWS, etc.) and the exact steps you followed, in order to help us reproduce what failed for you?

Earlier, I used master, and AWS_PROFILE to configure a non-default profile with MFA, but couldn't reproduce the initial problem you had:

I'm trying to create a cluster with using a specific profile. I've set the AWS_PROFILE environment variable (as per this) but its still creating the cluster using the default cli profile.

[richardcase]

@marccarre - sure no problem. Let me know if you need anything in addition to what i've included below.

eksctl version: {"gitTag":"0.1.0-alpha.6", "builtAt":"2018-06-08T11:31:30Z", "gitCommit":"1ed92fe1b1369d5c824cc96e5cab83887e2fc2b3"}

env vars: no AWS_* environment variables set

My local AWS config file:

[profile default]
region=eu-west-1
output=json
[profile dev]
region = eu-west-1
role_arn = arn:aws:iam::ACCOUNTB:role/dev
source_profile = default
mfa_serial = arn:aws:iam::ACCOUNTA:mfa/rcase

And my credentials file:

[default]
aws_access_key_id = ABCDEFGH
aws_secret_access_key = ABCDEFGHIJKLMNOP

And the step to reproduce:

1. Ensure there are no AWS_* envionment variables set
2. Run the following:

AWS_PROFILE=dev eksctl get clusters --verbose=4

This will give the following error:

2018-06-13T16:07:26+01:00 [✖]  checking AWS STS access – cannot get role ARN for current session: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, dev.
EC2RoleRequestError: no EC2 instance role found
caused by: RequestError: send request failed
caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials: dial tcp 169.254.169.254:80: connect: host is down%  

If i use the aws cli in a similar way it correcly picks up the profile and asks me for my MFA code:

AWS_PROFILE=dev aws ec2 describe-security-groups
Enter MFA code for arn:aws:iam::ACCOUNTA:mfa/rcase:
{
    "SecurityGroups": [
        {
            "Description": "Security group for nodes",
            ....

[marccarre]

Thanks a lot @richardcase, I was able to reproduce your issue (same error message/symptoms) with both master and 0.1.0-alpha.6. Testing against the PR you opened, I still had:

$ AWS_PROFILE=dev ./eksctl get clusters --verbose=4
Assume Role MFA token code: XXXXXX
2018-06-13TXX:XX:XX [✖]  checking AWS STS access – cannot get role ARN for current session: AccessDenied: User: arn:aws:iam::XXXXXXXXXXXX:user/xxx is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::XXXXXXXXXXXX:role/xxxxxx
    status code: 403, request id: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

Is there any additional configuration step you did in between, for this to just work?

[richardcase]

@marccarre - the error you are getting is because your user is not authorised to assume the role you specified. This will be an iam issue as this error is being returned from aws.

You should get the same error if you tried to assume role using the aws cli. For example:

aws sts assume-role --role-arn "arn:aws:iam::XXXXXXXXXXXX:role/xxxxxx" --role-session-name "RoleSessionTest" 

[marccarre]

Indeed, I had misconfigured that role. Once sorted out, I was able to get things to just work:

$ AWS_PROFILE=dev ./eksctl get clusters --verbose=4
Assume Role MFA token code: XXXXXX
2018-XX-XXTXX:XX:XX [▶]  role ARN for the current session is "arn:aws:sts::XXXXXXXXXXXX:assumed-role/XXXXXXXXXXX/XXXXXXXXXXXXXXXXXXX"
2018-XX-XXTXX:XX:XX [▶]  clusters = {
  Clusters: ["foo","bar"]
}
2018-XX-XXTXX:XX:XX [▶]  cluster = {
  Cluster: {
    Arn: "arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/foo",
    CertificateAuthority: {
      Data: "XXX"
    },
    [...]
    Status: "ACTIVE",
    [...]
  }
}
[...]

[marccarre]

Fixed by #59. And change released in 0.1.0-alpha.7.

[NicholusMuwonge]

Hey guys, I was getting the same issue, the funny thing is, I had to just adjust computer date and time and it was back on the road. So please try this out and let me know if it helps.

[fd98279]

export AWS_DEFAULT_PROFILE=...
worked for me

[peringangichandrakala]

Hi all,
I am trying to create Kubernetes cluster using eskctl, I am facing problem as shown below while creating cluster

could anybody help me in solving this

[peringangichandrakala]

Please crate iam role for eks may be it will help you in resolving
this issue. And after creating iam role once again do aws configure and
mention all the details.

Regards,
Chandrakala

On Thu, Jul 9, 2020, 9:26 AM Harshwardhan Kakra [e-mail]
wrote:

Hi all,
I am trying to create Kubernetes cluster using eskctl, I am facing problem
as shown below while creating cluster
[image: Screenshot (20)]
https://user-images.githubusercontent.com/50578841/78328898-eb868b80-759d-11ea-80eb-00c93db9a1dd.png

could anybody help me in solving this

i'm also facing this same error

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/weaveworks/eksctl/issues/57#issuecomment-655883510,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMB4LGOZKDCQVP5DHP3H5ADR2U5WFANCNFSM4FEE5ZNA
.