kops creates only 2 nodes and no master

[bpasson]

kops version: Version 1.7.1 (git-c69b811)
kubectl version: 1.8.2
kubernetes version: 1.7.4
cloud provider: AWS

When following @arun-gupta workshop kubernetes-aws-workshop, executing the following command gave only 2 nodes and no
master.

kops create cluster \
  --name example.cluster.k8s.local \
  --zones $AWS_AVAILABILITY_ZONES \
  --yes 

I was expecting to have 2 nodes and 1 master. A work around is to specify explicit node-count and master-count options.

I used a newly created AWS account.

[chrislovecnm]

I am guessing that the az does not have m3 nodes. Can you verify that the ASG was created does not have errors?

[chrislovecnm]

@arun-gupta I noticed that you are using eu as a region in the guide, let me double check that comment. I would recommend us-east-2 or us-west-2.

[chrislovecnm]

Sometimes the cluster creation does not work and the validation fails. This happens when only worker nodes are created and master node is not provisioned. This is filed as kops/#3751. As a workaround, specifying the exact number of master node(s) and worker node(s) will create the cluster successfully.

This is not correct for the regions that I mess with all the time. If the ASG is created kops did its job, and aws is not allowing the instance to be created. Probably add a note to check the asg.

[arun-gupta]

I executed the same command and it created the cluster fine in my setup. @bpasson created the AWS account last week. Could it be related to that?

eu-central-1 has the three zones: eu-central-1a, eu-central-1b, eu-central-1c. These are the ones used here.

IMHO kops should perform validation that the instances can be created or not in a particular region, and then fallback on a different instance type if it could not be created.

[chrislovecnm]

@arun-gupta we are doing the validation that is possible, for instance types, but we are unable to validate a users quota on instances. Maybe the code for validation is not quite working correctly still. I need to understand what the issue @bpasson is having. Need more context and data.

Would be nice if we improved kops cluster validate so that checks the ASGs.

I would ask that a note is added to the guide to check the ASG, or we use c4.large, or t2.large instances for the masters.

The workaround may be to specify the instance type for master.

[arun-gupta]

@chrislovecnm I thought kops would fallback on available instances in a region. This makes the experience more seamless for user as opposed to explicitly requiring to specifying an instance type.

[chrislovecnm]

We check the instance type via the aws API ;)

[chrislovecnm]

If one of your aws gurus can recommend something better than this https://github.com/kubernetes/kops/pull/2837

We are happy to have a contribution ;)

We can improve kops validate to respond if an asg is having problems, I think. But from a pre-create asg standpoint, we are using the above code to check for the available instance type.

[andresguisado]

@chrislovecnm, I'm using an AWS account which doesn't have m3 instances available to choose running on London AWS region (eu-west-2a and eu-west-2b) and kops create cluster as @arun-gupta worked for me. it picked up a m4 instance type.

Was the initial problem the non-existence of m3 instances or it could be anything else?

Thanks

[chrislovecnm]

I believe that the initial problem was the lack of m3 instances. We need to determine what instance type we want for masters as m3’s are getting rare and sometimes do not exist at all.

[geojaz]

Figuring out which instances to use in each region is turning out to be a real pain... I'm only writing the below because I keep seeing or hearing about people having the wrong number of nodes coming up... it's apparently not an obvious thing. So we should try to make it better.

My presumption of the flow is that while creating the cluster, it's possible to create the ASG just fine with or without the instances being available in that region. (I could be completely wrong about how this works). If so, the instance type not available error would be on the ASG? If so, I don't really see a clean way to bounce back when instances aren't available.

I guess that's probably why we tell people to validate cluster. Assuming it's just a node problem, validate could potentially give some useful information. When it's a master, well- would it be worth leaving a friendly note after kops update to check the status of your ASGs if things don't come up as expected? Do we have a basic troubleshooting doc? I couldn't find one but I can think of a few things that should go in there.

[iwasnobody]

I also find the same exception in AWS BJS region, I had copy AMI and specify the Region/AZ information， when I create the cluster , only nodes can be setup and no master at all.
I set the master instance type as t2.medium but still the same problem. Pls give me a help

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close