Flux: Consider putting ssh public key in flux-git-deply kubernetes secret

[isen-ng]

Consider also putting the ssh public key of the deploy key (aka flux-git-deploy) in a kubernetes secret. Maybe even in the same secret.

If that is done, then users no longer has to depend on pod logs or fluxctl to get the identity.

I am coming from trying to deploy flux using helm on terraform. I could use terraform kubernetes provider to fetch secrets and the public key can be fetched like this.

Currently, I'm using a work around which fetches the private key, feed it into a external data source that contains a shell script to convert the private key into a public key.

So, also having the public key available is a nice to have (which will also preserve the public key comments) but isn't a blocker.

On this topic of ssh keys, it would also be nice if we could instruct helmOperator to generate a new git ssh key upon creation instead of specifying a secret name to point to. Then we can fetch the ssh public key using kubernetes secrets too. Users will no longer need extra steps to create the secret before using it.

[isen-ng]

I haven't written in go before, so the syntax might be totally wrong but a simple change like this should do it right (with related tests)?

https://github.com/weaveworks/flux/blob/master/cluster/kubernetes/sshkeyring.go#L119

    patch := map[string]map[string]string{
        "data": map[string]string{
            "identity": base64.StdEncoding.EncodeToString(privateKey),
            "identity.pub": base64.StdEncoding.EncodeToString([]byte(publicKey.Key))
        },
    }

[robertgates55]

@isen-ng - fully agree with your suggestion (🙏) but in the meantime any chance of sharing your terraform/shell that fetches, converts & outputs?!

[isen-ng]

data.tf

data "kubernetes_secret" "flux_identity_private" {
  # this line is required to force this to become a computed data source
  # this is because before flux is deployed, refreshing this data source
  # will complain that this secret does not exist, which is true, because 
  # flux is not deployed yet
  depends_on = ["helm_release.flux"]

  metadata {
    name      = "flux-git-deploy"
    namespace = "default"
  }
}

data "external" "flux_identity" {
  program = ["sh", "${path.module}/files/get-ssh-public-key.sh"]

  query = {
    ssh_private_key = "${data.kubernetes_secret.flux_identity_private.data["identity"]}"
  }
}

output "flux_ssh_public_key" {
  value = "${data.external.flux_identity.result["ssh_public_key"]}"
}

get-ssh-public-key.sh

#!/usr/bin/env bash

set -e

eval "$(jq -r '@sh "export PRIVATE_KEY=\(.ssh_private_key)"')"

echo "$PRIVATE_KEY" > /tmp/ssh_private_key
chmod 600 /tmp/ssh_private_key

ssh-keygen -f /tmp/ssh_private_key -y > /tmp/ssh_public_key

SSH_PUBLIC_KEY=$(cat /tmp/ssh_public_key)

rm -f /tmp/ssh_private_key
rm -f /tmp/ssh_public_key

jq -n --arg ssh_public_key "${SSH_PUBLIC_KEY}" '{"ssh_public_key":$ssh_public_key}'

Do make sure your /tmp mount is of tmpfs type so that you actually avoid writing the private key to disk.

[rseedorff]

Hi @isen-ng

if found another possible solution for your problem using terraform to deploy flux on a cluster fully automated. You can create a new ssh key in terraform and deploy this ssh key as kubernetes secret flux-git-deploy first before deploying flux:

resource "tls_private_key" "flux" {
  algorithm = "RSA"
  rsa_bits  = 2048
}

# Creating a new namespace for flux to store the ssh key secret upfront
resource "kubernetes_namespace" "flux" {
  metadata {
    name = local.flux_namespace
  }
}


# Creating a new kubernetes secret with the new private key. This should be used by flux to identify itself against git later on
# see Docs: https://docs.fluxcd.io/en/1.18.0/guides/provide-own-ssh-key.html
resource "kubernetes_secret" "flux-git-deploy" {
  metadata {
    name      = "flux-git-deploy"
    namespace = local.flux_namespace
  }

  type = "Opaque"

  data = {
    identity = tls_private_key.flux.private_key_pem
  }
}

# Add a deploy key to the corresponding gitlab flux project
resource "gitlab_deploy_key" "flux_deploy_key" {
    title = "Flux deploy key ${var.cluster_name}"
    project = data.gitlab_project.flux_project.id
    key = tls_private_key.flux.public_key_openssh
    can_push = true
}

Later on if you deploy flux via terraform also (depends on the generated secret key) you can reference the already created k8s secret and flux will use this key instead of creating a new one.

data "helm_repository" "fluxcd" {
  name = "fluxcd"
  url  = "https://charts.fluxcd.io"
}

resource "helm_release" "flux" {
  name      = "flux"
  namespace = local.flux_namespace

  repository = data.helm_repository.fluxcd.metadata[0].name
  chart      = "flux"

  set {
    name  = "git.url"
    value = var.git_url
  }

  set {
    name  = "git.secretName"
    value = "flux-git-deploy"
  }

  set {
    name  = "syncGarbageCollection.enabled"
    value = var.flux_garbage_collection_enabled
  }

  set_string {
    name  = "ssh.known_hosts"
    value = var.git_known_host_key
  }
}

resource "helm_release" "helm-operator" {
  name      = "helm-operator"
  namespace = "flux"

  repository = data.helm_repository.fluxcd.metadata[0].name
  chart      = "helm-operator"

  set {
    name  = "git.ssh.secretName"
    value = "flux-git-deploy"
  }

  set {
    name  = "helm.versions"
    value = "v3"
  }
}

[cf-sewe]

I justed wanted to leave a quick thank you @rseedorff / @isen-ng for the detailed examples above. It's nice to see other people having the same issues, so I know I am not completely off-track ;-)

[stefanprodan]

In Flux v2, when you generate the SSH keys with tk, it creates a secret that contains the private key, public key and known hosts. https://toolkit.fluxcd.io/components/source/gitrepositories/#spec-examples