Pulumi-kubernetes: Allow tranformations to omit or filter

[rosskevin]

Currently a transformation can only alter the object. Given the ordering of code:
https://github.com/pulumi/pulumi-kubernetes/blob/a610ed8cf87e147aa4b8530a8eb79a201d0468ca/sdk/nodejs/provider.ts#L2081-L2084

if all object keys were to be removed, it would cause a validation error.

A couple of suggestions:

1. change the transformation api to allow a return value to be interpreted; or
2. change ordering of code to allow all keys to be removed, causing [] to be returned (like https://github.com/pulumi/pulumi-kubernetes/blob/a610ed8cf87e147aa4b8530a8eb79a201d0468ca/sdk/nodejs/provider.ts#L2077-L2079)

use case

Installing cert-manager from manifests but want to alter the namespace (and the raw manifest contains a namespace declaration).

Potential usage (not considering potential better variations to the transformation signature):

    const certManager = new k8s.yaml.ConfigFile(
      'https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/cert-manager.yaml',
      {
        transformations: [
          createTransformAddNamespace(namespace),
          // Remove the namespace declaration.
          (obj: any) => {
            if (obj.kind == 'Namespace') {
              return null
            }
          },
        ],
      },
      { parent: this },
    )

[hausdorff]

This is a good idea, and one we will probably solve in the course of standardizing on a transforms API across Pulumi. That said, there are tradeoffs for various transforms APIs which are not obvious, and I'm not yet sure how to balance them -- @lblackstone @lukehoban when we go to solve these, let's all chat, I think I probably have the most context on those requirements.

[hausdorff]

@pgavlin I think we should pick either this or #161, and settle on the "real" transforms API for Q3.

[hausdorff]

Let me amend what I said before—#161 will continue to track the "finalize the transformations API". This issue will be scoped to the specific issue of allowing the transformations API to omit resources.

We will adopt @rosskevin's proposal and take null to signal that the object should be omitted.

[redbaron]

as a workaround one can "transform" object into a v1.List with no items:

tranformations: [
             (o: any) => {
                o.apiVersion = "v1";
                o.kind = "List";
            },
]

Edit: Simplified workaround

[Jean-Baptiste-Lasselle]

Hi, I have a use case to describe, that is pretty valuable, because will apply to many preject teams :

• many apps are today deployed in Kubernetes using Traefik as ingress Controller
• traefik recenty had a breajking change :
  
  
  
  • apps that are deployed in the cluster, must now define a completely new type of Ingress
  
  
  • many such apps have their own helm charts, often including a good old deprecated Ingress
  
  
  • So that I always end up with those apps, wondering : How do I just completely remove the Ingress Resource, I don't need to modify it, just remove it.
  
  
  • And then I deploy Ingress Routes, independently from, and subsequently to the execution of the apps' Helm Chart
  
  

just to mention Traefik is included in the official _"curated"_ helm charts : it a wide adopted ingress controller for k8S

[lblackstone]

• How do I just completely remove the Ingress Resource, I don't need to modify it, just remove it.

https://github.com/pulumi/pulumi-kubernetes/issues/486#issuecomment-527958122 will allow you to remove a resource. You just need to wrap the inner logic in a conditional to check the GVK and name.

[lblackstone]

I took another look at this, and don't think the proposed approach will work for all SDKs. NodeJS distinguishes between null and void function return values, but Python does not (always returns None). I don't see a backward compatible way of supporting this in the existing transformations API.

Since it's possible to omit resources with this workaround, I think it's best to leave the behavior unchanged, and document the workaround in the resource docs.

[iridian-ks]

@lblackstone generally using null/None is bad practice anyways so maybe it's a blessing that there's inconsistency. I did try to use del obj in Python to delete the object, but it seems the resource was already sent to the server so it didn't do anything.

It does look like there is also a transformations member for ResourceOptions. It looks like it leverages a particular return type of ResourceTransofrmationResult. I'm wondering if this could be used as an alternative to the Kubernetes specific transformations? Should there potentially be a change in pulumi proper to maybe extend ResourceTransformationResult to flag the resource to be removed from a preview? Something like:

return ResourceTransformationResult(delete=True)

That should be portable between all languages/providers, I think, and doesn't have any weird hacks.

[lblackstone]

I'm wondering if this could be used as an alternative to the Kubernetes specific transformations?

Yeah, we might be able to standardize this in the future, but it would be a breaking change. The k8s transformations API was developed before the ResourceOptions one existed, hence the bespoke API.

[lblackstone]

A transformation to omit resources is now documented for every SDK, so I'm going to close this out.