Pulumi-kubernetes: Helm 3 not installing custom CRDs

[brpaz]

Problem description

I am trying to install cert-manager with Helm and Pulumi.

I have this configuration:

const certManagerChart = new k8s.helm.v3.Chart(
  "cert-manager",
  {
    chart: "cert-manager",
    repo: "jetstack",
    version: "v0.13.0",
    namespace: certManagerNs.id,
    values: {
      ingressShim: {
        defaultIssuerName: "letsencrypt-prod",
        defaultIssuerKind: "ClusterIssuer"
      }
    }
  },
  { providers: { kubernetes: provider }, dependsOn: cluster }
);

The custom CRDs provided by the chart are not being installed. I have exactly the same setup in Terraform and it works, so I guess it´s an issue of Pulumi Helm resource and not the chart.

If I understood correctly Pulumi uses helm template under the hood. I think the issue might be that the command doesn´t allow specifying the "include crds" flag. (see https://github.com/helm/helm/pull/7138/files)

[nesl247]

Just ran into this myself. It is expected that Pulumi would install the CRDs automatically the same as helm would.

I think the big fix would be to use the 1.0.0 of the terraform-provider-helm.

[lblackstone]

I think the issue might be that the command doesn´t allow specifying the "include crds" flag.

Yes, this is correct. We'd need to plumb that option through to the SDK.

[rhyek]

Could this be related to #993? In my case it's not installing any resources at all.

[brpaz]

@rhyek I don´t think so. This issue is related with Custom CRDs and the reason is well identified by the missing support for the "include crds" flag.

I have installed like 5 or 6 Helm charts into my cluster without any problem.

[dz-pyps]

I can confirm that with pulumi v2.0.0 and pulumi-kubernetes==2.0.0 (Python SDK), CRDs are not installed. I tried to install the traefik chart https://github.com/containous/traefik-helm-chart/tree/master/traefik

[jaxxstorm]

We're going to add this to the provider, but if you come here and you're trying to install a chart with CRDs, a workaround is to install them using ConfigFile. As an example (in typescript):

const crds = new k8s.yaml.ConfigFile("crds", {
    file: "https://github.com/jetstack/cert-manager/releases/download/v0.14.1/cert-manager.crds.yaml",
}, { provider: provider })

const certManager = new k8s.helm.v2.Chart("cert-manager",
    {
        namespace: namespace.metadata.name,
        chart: "cert-manager",
        version: "v0.14.2",
        fetchOpts: { repo: "https://charts.jetstack.io" },
        values: {}
    },
    {
        providers: { kubernetes: provider },
        dependsOn: [crds]
    },
)

[dz-pyps]

@jaxxstorm, I did precisely that. Nevertheless, as you can imagine, it is inconvenient to download the CRDs locally and keep track of the changes in the target repository.

crds_pulumi_objects = []
for item in glob.glob(traefik_cdrs_directory + '/*'):
    if '.yaml' in item:
        crd_to_apply = ConfigFile(
            'traefik_crd_' + item.split('/')[-1].split('.')[0],
            item,
            opts=ResourceOptions(
                depends_on=[
                    traefik_namespace,
                ],
                provider=k8s_provider,
            )
        )
        crds_pulumi_objects.append(crd_to_apply)

Output.all(
    traefik_dashboard_out_file_location,
    traefik_dashboard_j2_location,
).apply(pulumi_output_renderer_traefik)

apply_traefik_ingress_route = ConfigFile(
    'traefik_ingress_route',
    traefik_dashboard_out_file_location,
    opts=ResourceOptions(
        depends_on=[
            traefik_namespace,
            *crds_pulumi_objects,
        ],
        provider=k8s_provider,
    )
)

[jaxxstorm]

This should be available in the next release of the provider!

[dz-pyps]

@jaxxstorm, I've updated pulumi-kubernetes to 2.2.0, but CRDs are still not installed by default.
I used traefik chart https://github.com/containous/traefik-helm-chart

[jaxxstorm]

@jaxxstorm, I've updated pulumi-kubernetes to 2.2.0, but CRDs are still not installed by default.
I used traefik chart https://github.com/containous/traefik-helm-chart

which version of helm are you using locally?

[dz-pyps]

@jaxxstorm version.BuildInfo{Version:"v3.2.1", GitCommit:"fe51cd1e31e6a202cba7dead9552a6d418ded79a", GitTreeState:"clean", GoVersion:"go1.13.10"}

[dz-pyps]

@jaxxstorm, CRDs from this chart https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack do not get installed first, so dependant resources fail.