Zero-to-jupyterhub-k8s: OAuth and env secrets in config.yaml

[betatim]

Should things like https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/d6340ab6102747103d4bb58af729c104c39f8117/jupyterhub/templates/hub/configmap.yaml#L18 be stored as a secret instead of a configmap? I looked into this while splitting my config file into config.yamls and secrets.yaml and moving things from the config to secret. The idea is that only the latter needs to be encrypted/not stored in GitHub.

Related to this, I have to set some ENV variables via hub.extraEnv (e.g. JUPYTERHUB_CRYPT_KEY) that I'd rather keep secret. Is there a way to merge hub.extraEnv from a.yaml and b.yaml when running helm upgrade ...? Or would we have to add a hub.secretExtraEnv that is concatenated with hub.extraEnv in a template?

[consideRatio]

@betatim Whats stored in a Kubernetes Secret object and a Kubernetes ConfigMaps object is determined by the templates in the helm chart. The templates are generated using values.yaml and other value files, so no matter if you store sensitive stuff in a Kubernetes Secrets, you also need to be able to create the Secret from values.yaml (or hardcode it inside the templates describing the Secret template).

For example, this secret refers to sensitive information provided as a value rather than having it hardcoded in it. So how would you protect your sensitive information, provide the value at the same time as sharing your code on GitHub?

What I have done, is that I use git-crypt to encrypt a file i named secret-values.yaml, and I always pass it to the helm upgrade command. It would then override/merge certain yaml values such as auth.google.client-id for example. git-crypt makes sure that whatever is pushed to github in secret-values.yaml is encrypted, and whenever i pull it becomes unencrypted.

My code is based mainly on @yuvipanda 's work in various repositories and this repo.

[yuvipanda]

I think there are two issues here:

1. Some secrets that should be in kubernetes Secret objects are instead in ConfigMap objects
2. How to split config.yaml into multiple files properly to preserve secrets

(1) is a bug we should fix.

For (2), I believe helm will just merge 'dictionaries' like extraEnv together, so you can specify them both in your config.yaml and your secrets.yaml and it should be ok.

Also there's native support for enabling auth_state in https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/master/jupyterhub/values.yaml#L112. It's not documented anywhere mostly because there's no clear way to use that yet, but you can set your secret key there.

[betatim]

For (2), I believe helm will just merge 'dictionaries' like extraEnv together, so you can specify them both in your config.yaml and your secrets.yaml and it should be ok.

I thought that too but when I tried it all I got was the value from the second file. Not sure what the right words to type into a search engine are to find out what the docs say. Alternatively, is there a command to get helm to show me the "generated yaml we will use i you were to run this command"?

Also there's native support for enabling auth_state in https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/master/jupyterhub/values.yaml#L112. It's not documented anywhere mostly because there's no clear way to use that yet, but you can set your secret key there.

I completely missed that! Thanks for the 👀.

[yuvipanda]

yep. you can use '--debug' (and --dry-run with it if you don't want it to
execute) to see what is up

On Tue, Jan 23, 2018 at 11:34 PM, Tim Head [e-mail] wrote:

For (2), I believe helm will just merge 'dictionaries' like extraEnv
together, so you can specify them both in your config.yaml and your
secrets.yaml and it should be ok.

I thought that too but when I tried it all I got was the value from the
second file. Not sure what the right words to type into a search engine are
to find out what the docs say. Alternatively, is there a command to get
helm to show me the "generated yaml we will use i you were to run this
command"?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/440#issuecomment-360045456,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB23gwGDiiHBosu9BDhe-y-XSGYhSYUks5tNt0UgaJpZM4RpqxJ
.

--
Yuvi Panda T
http://yuvi.in/blog

[betatim]

I put the below in a secrets.yaml and ran helm upgrade deployment jupyterhub/jupyterhub -f config.yaml -f secrets.yaml:

auth:
  state:
    enabled: true
    cryptoKey: "123456789...."

When I then kubectl describe pod the hub I don't see JUPYTERHUB_CRYPT_KEY in the environment and using get_auth_state() returns None. Typo in my yaml or ...?

My config.yaml looks basically like this.

[consideRatio]

Hmmm, if we mount a secret and a configmap, and both writes stuff to the file system - will they overwrite each others values at times or something like this?

We should address the complexity of having the hub configmap as well, perhaps we can solve this issue if we do that. Currently the configmap is only a path for the helm template values to propegate towards the hub containers jupyterhub_config.py more or less.

[manics]

Here's a workflow where having secret values handled as secrets would be beneficial.
I'm using helmfile which supports diff and a related command apply https://github.com/roboll/helmfile/issues/458.

helm diff includes a --suppress-secrets arg that hides secrets from the displayed diff.

A couple of ideas:

• Have a switch that changes all of jupyterhub/templates/hub/configmap.yaml to be a secret
• Allow all config keys to be defined under a new toplevel key secret, and merge the two, looks like merging would already be handled: https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/580f46ef44f3b1ca0feedf0a38d26c712f51f8cf/images/hub/z2jh.py#L21

For example, if this was provided it would be stored in the configmap:

auth:
  type: github
  github:
    clientId: CLIENT-ID
    clientSecret: CLIENT-SECRET

whereas this would be stored in the secret:

secret:
  auth:
    type: github
    github:
      clientId: CLIENT-ID
      clientSecret: CLIENT-SECRET

[koen92]

@manics yes indeed, by combining config map and secret in this way, we don't need to specify them anymore in config.yaml.

The only thing that is missing, is the support for using existing secrets, which I added with https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/1142.