Aws-cdk: [aws-eks]: Deploy Helm chart to EKS via CDK

[zanhsieh]

:rocket: Feature Request

General Information

NA

Description

Helm chart is a de facto solution while deploying 3rd party solutions into Kubernetes. In general, developers would like to have those solutions ready, otherwise they need to spend huge amount of time for mocking up those 3rd party solutions (a blackbox per se). If we could deploy 3rd party solutions into EKS with CDK, I feel it will attract even more developers to work on their solutions within AWS EKS.

CDK might consider this as a plugin to support since we all know Helm api is not maintained by CDK.

BTW, Terraform is able to deploy Helm chart.

[pahud]

@zanhsieh I am the maintainer of aws-lambda-layer-kubectl, which has been the core component of @aws-cdk/aws-eks
https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-eks/lib/kubectl-layer.ts

I believe it's possible as long as we build the helm client into a standalone lambda layer so CDK can create yet another construct lib on top if it.

I'll look into it and any thoughts would be appreciated.

[zanhsieh]

Got some idea from this repo:

https://github.com/microsoft/helm-web-api

The repo above still works today. So the basic workflow (although it might be a lot of steps) to make this work is:

• Rip off tiller-deploy deployment and service yaml files and use CDK to deploy while creating the cluster (any existing cluster with helm shall have these two files)
• Build docker image (on-demand-micro-services-deployment-k8s) and push to some repo (your own private or public repo like docker hub) according to helm-web-api README.md
• Modify the chart values.yaml registry line, point to your repo.
• Grep the yaml manifest files by calling --debug and --dry-run with:

helm install chart/on-demand-micro-services-deployment-k8s --name on-demand-micro-services-deployment-k8s --set rbac.create=true

• Deploy previous steps yaml files with CDK while creating the cluster

After deployment, just follow the syntax as helm-web-api README.md to deploy any helm chart via code.

Yeah, it's messy and hacky, and it's neither M$ way nor AWS way ...

[eladb]

I think @pahud's approach makes senes. If he can add the Helm client in the kubectl layer, we could create a custom resource that interacts with the client and creates/updates/deletes Helm charts from your cluster.

[abelmokadem]

Saw that helm was recently added to the lambda layer. Thanks @pahud. Looking forward to see helm support in CDK so we can cleanup our bash scripts.

[eladb]

Yes this will be awesome to support.

[Douglas-Scott]

A clear example of supporting helm with a CDK deployment would be helpful. Thanks for all the awesome work!

[FarshadNiayesh]

Hi @pahud, I get this error when trying to add helm to a kubernetes cluster in python with both the construct eks.helm_chart() and cluster.add_chart()

Failed to create resource. Error: b'Release "nonprodrptclusternonprodchartnginxingress0406e3cf6676" does not exist. Installing it now.\nError: create: failed to create: namespaces "nginx" not found\n' at invokeUserFunction (/var/task/framework.js:85:19) at process._tickCallback (internal/process/next_tick.js:68:7)

This is my code:

eks_cluster.add_chart(id="nginx-ingress-0406", chart="nginx-ingress", namespace="nginx", repository="https://kubernetes-charts.storage.googleapis.com", wait=True)

eks_cluster.add_chart(id="airflow-0406", chart="airflow", namespace="default", repository="https://kubernetes-charts.storage.googleapis.com", wait=True)

Any ideas?

[pahud]

@FarshadNiayesh

I am afraid the vended Helm3 will not create new namespace unless you specify --create-namespace?
https://github.com/helm/helm/issues/5753
https://github.com/helm/helm/pull/7648
https://github.com/helm/helm/issues/6794

But it makes sense to bump the Helm3 version to the latest v3.1.2

Please watch https://github.com/aws-samples/aws-lambda-layer-kubectl/issues/27 and I will bundle the latest Helm v3.1.2 in the kubectl layer before creating a PR for CDK to improve this.

[FarshadNiayesh]

Thank you so much for your response!