Kops: How to create a cluster from a file?

[WillemMali]

I want to version my clusters and not be forced to declare my clusters in CLI arguments in a Makefile. I see there is a file parameter to kops create cluster, but I can't for the life of me find documentation for it, I don't have any idea what this file's format is except that it's probably YAML.

I have the same problem with kops rolling-update cluster.

Do we currently support declaratively managing clusters? Why does only kops edit seem to expose the cluster definition? Would it be hard to implement a two-stage equivalent to kops edit, where you first export the config, put it in source control, edit it, and send it to kops for deployment with rolling update?

How I would imagine the workflow:

$ kops export cluster mycluster.com >cluster.yaml
...
$ vim cluster.yaml # edit the cluster at your leisure, version it etcetera
$ kops import -f - <cluster.yaml
$ kops rolling-update

Instead of:

$ kops edit cluster mycluster.com
$ kops update

[WillemMali]

I'm very much willing to implement this myself, but I think I would require a not insignificant amount of guidance as I'm a Go newbie and because I am not yet very familiar with the kops codebase. I prefer whatever option gets me up and running with a workflow like this quicker.

[mikesplain]

I'd be very interested in this as well. I like the fluidity of using kops to make instancegroups, etc. and test but it does make it hard to apply successful changes to other environments. That said, I feel like this is kind of why there's a terraform output... not sure what the right middle ground would be.

[mad01]

The idea is something that i have been thinking of as well. The use case i see it to have a minimal template the is a bit richer then what you get in the cli to apply like the company cluster template. but on the other hand you can accomplish close to that with just the cli to and a example to.

i like the way it's currently working with the state store in S3

[chrislovecnm]

The real problem is that using cluster spec YAML is not documented well :(

I have a PR in that automates the process that does the export in one command, but create a cluster on the command line to get the state store setup, and use this:

# $1 is your cluster name
kops get cluster $1 -o yaml > $1.yaml
echo " " >> $1.yaml
echo "---" >> $1.yaml
echo " " >> $1.yaml
kops get ig --name $1 -o yaml >> $1.yaml

I have a PR in that does the above two steps.

From there you have a YAML file where you have full control to modify any of the options. All of the ClusterSpec is completely open. See here https://godoc.org/k8s.io/kops/pkg/apis/kops

Once the YAML is updated kops replace -f mycluster.yaml replaces the YAML in the state store, then kops update.

We also have kops create -f which accepts a file. Next step after that is to create the ssh secret and then run kops update. We have an issue in to have kops create -f my.yaml -i myssh.pub --yes to work, and it is three steps currently.

[WillemMali]

I think it would also be nice to be able to do something like this:

$ kops create cluster --dry-run -o yaml
````

This would effectively be a boilerplate generator, and be closely analogues to:

$ kubectl run myapp --image=myimage:latest --dry-run -o yaml
$ kubectl create deployment --dry-run -o yaml

Then you could create/update the cluster/instancegroup with:

kops apply -f - ```

That would be very slick IMO. I think it would be great to make the kops CLI capable of workflows very similar to those available with kubectl, so it feels more like you're working with "just another Kubernetes object/controller".

I think I may be capable of building this right now, but I don't know how we could make the cluster spec self-documenting. Could someone who's more familiar with the Go ecosystem look at that, what library to use etc. and then open an issue for it?

[chrislovecnm]

So the replace method we have should probably be aliased as apply https://github.com/kubernetes/kops/blob/master/cmd/kops/replace.go

It does the replace as you mentioned.

The docs website has a new section that the seem to be using something maybe a nodejs tool to generate docs off of the Kubernetes main API. Someone should look at that.

The

kops create cluster --dry-run -o yaml

Is already a filed issue that would be awesome!

kops init cluster -o yaml --name foo --zones us-east-1

Is probably the way to go.

Folks have also discussed also havin a helm like templating functionality, but that is down the line.

[WillemMali]

Why do you say kops init cluster is the way to go over e.g. kops create cluster, or am I not understanding you right?

I think the latter better matches the CLI of Kubernetes, and I don't see the value add in separating the two into create (generate template and make cluster) and init (generate template and do nothing).

[chrislovecnm]

Init is the only thing that is not implemented. There is much value in generating a template since many users only use yaml. Many advanced options exist that are only accessible via yaml.

[WillemMali]

Isn't init already implemented in the form of kops create cluster -o yaml without --yes? I think I did that today, and though it's still a
bit more cumbersome than with k8s (because I have to deal with the
cluster not existing (yet) when I try to update it) it's already pretty
good.

I currently use yaml files (not file, many-in-one file doesn't seem to
work properly) with environment variables ($ENV) and envsubst to assemble
my clusters, and so far I like it a lot. I have a *.yaml.template file
for parameterization/deployment generation (which includes the $ENV
variables), and a script which converts it to *.yaml files, which is a
dependency of my deploy Makefile target.

On Sun, May 21, 2017 at 10:59 PM, Chris Love [e-mail]
wrote:

Init is the only thing that is not implemented. There is much value in
generating a template since many users only use yaml. Many advanced options
exist that are only accessible via yaml.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/kops/issues/2603#issuecomment-302962845,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEePF4WnajJ2Qs7Dn2qBXPwLmY3h-B-Cks5r8KW5gaJpZM4NgYsa
.

[chrislovecnm]

If it is implemented that is news to me ;)

[WillemMali]

Ah, it isn't, I worked around it by using kops create cluster and then kops get cluster -o yaml, and then deleting the cluster.

After that you can take down the cluster and get it up again with the generated yaml files (don't forget to poll for the cluster to come up using validate, this will break your update step), and use replace and a subsequent rolling update to imitate the kubectl apply flow.

[chrislovecnm]

That is why I would recommend 'kops init' which models helm init. Have the options for create or even kops create cluster init subcommand that would have all of the create cluster cli options already.

So
kops create cluster init -o yaml (we issues with json currently)
Kops create ig init

[WillemMali]

I think I prefer the kops init ig variant, seems more consistent with the other parts of the CLI. I haven't used helm yet so I can't speak to that.

[chrislovecnm]

With the subcommand you have access to all of the cli flags already. That is why I would recommend it. Just a thought

[WillemMali]

I don't think the internal CLI implementation should take priority for designing the command tree. I'll look into implementing an init command some time this week.

Perhaps we could go the other way and fill a Cluster directly from the CLI implementation instead of using the intermediate CreateClusterOptions?

[hypergig]

@chrislovecnm I am confused...

Once the YAML is updated kops replace -f mycluster.yaml replaces the YAML in the state store, then kops update.

Does replace not imply 'roll the whole entire cluster no matter how small the change, even if the cluster doesn't need a full roll'?

Is a kops replace -f mycluster.yaml like a non interactive kops edit? I assume not because you don't have to do an update.

[chrislovecnm]

@hypergig - kops replace -f myexample.yaml is what you are looking for. Also, take a look at https://github.com/kubernetes/kops/pull/2795

[chrislovecnm]

You might want to take a look at https://github.com/kubernetes/kops/blob/master/docs/manifests_and_customizing_via_api.md

Let us know if that documentation helps.

[chrislovecnm]

Closing this as docs are updated

[ms4720]

just to put my 2 cents in on naming commands, why not:

kops apply -f file.(yml|json) $NAME
and
kops apply -f file.(yml|json) --yes  $NAME

seems more in line with kubectl

[srflaxu40]

So, essentially this becomes a create and then replace (with existing YAMLs)? Also, are we sure this is exporting instance groups? I tried this with v1.8 and I only see the cluster manifest; no groups. https://github.com/kubernetes/kops/blob/master/docs/manifests_and_customizing_via_api.md

[ms4720]

kops 1.8.0 gives me instance groups and cluster with the following:

• kops create --dry-run ...
• edit file
• kops create -f file.yaml

gets a 3 nod and 3 master cluster up in aws, I need to say I am far from an expert in kops or kubernetes

verified with: kops get $NAME -o yaml , and it is up and running

[ms4720]

Small update, that does not export secrets and I have no federations to test with

[emilhdiaz]

I'm testing with kops 1.8 as well on Ubuntu and both kops create -f and kops replace -f only export the cluster config. Ironically I have the same exact kops version on an Ubuntu subsystem (Windows 10) and that exports both the cluster and instance group definitions.