Charts: Chart for Hashicorp's Vault.

[andrewhowdencom]

I want to have a go at getting this up on Kubernetes in the next few days anyway. I can do it as a chart, and submit it to incubator, there's any interest.

Image: https://github.com/hashicorp/docker-vault

Will be implemented in H/A with the etcd backend (if I can make that work); unsure how I'm going to unseal it at this point.

Let me know!

[lachie83]

Fantastic! I also have a chart and a bunch of ideas. I wouldn't bother with automating the unseal. Just document a way to do it

[andrewhowdencom]

I'm playing with this (: It's slow going, but I'll get it done. Progress tracked here:

https://github.com/littlemanco/charts/tree/ahowden_add-hashicorp-vault

[F21]

Just be aware that the docs mentions that H/A with etcd is currently broken: https://www.vaultproject.io/docs/config/index.html#ha_enabled

Would it make sense to make consul the default backend?

[lachie83]

I don't mind. You can use the consul chart in the incubator repo as a dependency. My thought was to actually make S3 the default as it's the easiest to implement.

[ipedrazas]

I've used kubernetes jobs before to do the unseal, but every company has a different way of doing it. So, not worth adding them. Using a controller/operator would be a much better way to handle it.

If we go with S3 we will be limiting the reach of the chart to the people running it in AWS. Using the Consul chart sounds like a more generic/universal approach.

[F21]

I saw a gist where they store the unseal keys in vault, and automatically unseal a sealed vault if there are unsealed vaults available: https://gist.github.com/ccampanale/e230ed3e96a978776404

Maybe it's worth a look to see if this might be feasible?

[ipedrazas]

@F21 but that will be only useful for people who run more than 1 vault... how to manage vault is a company policy (or should be). I think it's important to provide charts that "just work" making as little assumptions as possible. That's why I like the idea of a controller because you can use the default behaviour or extend it to cover your own use case.

I'm not suggesting to add it as part of this chart, btw 😄

[andrewhowdencom]

Unfortunately, this is blocked (for me) for likely the next 2 weeks. I will come back to it, but at this time I'm sidetracked with some other instrumentation work. Sorry <3

[F21]

I am wondering if it might be worth starting a vault-operator project similar to the etcd-operator to help operate Vault. For example, one of its tasks could be to set up root and intermediate PKI backends, issue certificates and then restart Vault to use those certificates. It can also rotate those certificates before they expire and reload Vault's config as required.

[andrewhowdencom]

That would be cool. I had intended to set up the root and PKI backends with an install-job, but I hadn't given any thought to rotating them.

It's unlikely I'd have the skills required do this though =\

[F21]

The etcd team has opened a PR to support etcd3 as a Vault backend: https://github.com/hashicorp/vault/pull/2168

The code will be maintained by the etcd team and will use the etcd3 client, so it shouldn't have any issues with running Vault in HA mode.

Perhaps it would be best to use etcd3 as the default backend after all (once the pr is merged), because it would be possible to leverage the etcd operator to simplify operations.

[lachie83]

Closing for now. Happy to review a PR for this

[kachkaev]

Hi guys,

Has anyone been able to create a chart for vault? If so, would be great if you share it here!

I came across vault and vault-ui just yesterday and I'd really love to deploy it to a k8s cluster one day. Getting familiar with all these things now, but hope that my humble experiments wont last forever!

[F21]

@kabakaev I think creating a chart for running Vault in development mode should be pretty trivial. However, to run Vault for production, an operator needs to be built to automate most of the setup processes securely.

[andrewhowdencom]

@F21 what are the tasks that you envision an operator should take care of? I still have plans to get around to this, but it's taken a back seat to other infrastructure work. It's not my full time (or indeed, part time) gig haha

[F21]

Hey @andrewhowdencom

I've been playing with some ideas in my head, but haven't really found the time to commit anything to code as I've been super swamped over the last few months. I think it would be a good idea to dig into the etcd-operator and prometheus-operator's source to see how opinionated operators should be and how they should work.

Off the top of my head, I envision a Vault operator to setup Vault at a minimum:

1. Configure and init Vault.
2. Initialize Vault and encrypt the unseal keys using GPG and send/return them to the user(s): https://www.vaultproject.io/docs/concepts/pgp-gpg-keybase.html#initializing-with-gpg
3. Ask the user(s) to unseal Vault.
4. Set up Root and Intermediate certificate authorities within Vault to generate a certificate for Vault. Not sure how opinionated this needs to be, whether we have an intermediate with a long lasting certificate just for Vault or have short-lived certificates and regenerate + reload Vault or something else.
5. Perhaps ask the user to store their unseal keys in Vault. This may not be recommended, but would be a good option for those who have multiple Vaults in HA and want to automatically unseal any sealed Vaults because they were restarted by Kubernetes.

In terms of backups, I believe the consensus is to use the backup tools provided by the underlying storage backend, so it should probably be deferred to the etcd-operator or whatever operator is managing the storage backend.

[kachkaev]

There are two ongoing PRs for vault and vault ui now, both by @jpds! 🚀
https://github.com/kubernetes/charts/pull/1397
https://github.com/kubernetes/charts/pull/1406

Feel free to share your thoughts there!