Kustomize: add genericGenerator

[sethpollack]

for example:

genericGenerator:
  resourcePath: cronjob.yaml
  fieldPathArgs:
  - path: metadata.name
    value: mycronjob
  - path: spec.schedule
    value: "* * * * *"

[Liujingfang1]

@sethpollack Can you add some explanation of meaning of each field? I also opened #122 for supporting extensible transformation. It seems to have some overlap with the motivation of genericGenerator you proposed.

[sethpollack]

Sure! This would be a transformation where you can take a base resource resourcePath and modify its fields by passing fieldPathArgs with the path to the field and the replacement value.

A good example would be cronjobs where I may want to create a bunch of them from the same base - just with different names, commands, and schedules.

The transformation would just loop through the provided fieldPathArgs rather than hardcoded PathConfigs and just call mutateField for each.

[Liujingfang1]

Is resourcePath a file path to a resource file in the base folder? Usually the overlays are not supposed to know the exact files or structs of the base.

[sethpollack]

yes

[Liujingfang1]

This feature looks promising to me. I am also thinking about some other use cases that it might support. Say I want to replace a container image name no matter it is in a deployment object or in a pod. How could I define the resource and field path so that it can work?

Can you write a brief design doc about this genericGenerator(maybe some other name genericTransformer)? Let have others' opinion on it.

[sethpollack]

I think the genericGenerator solves a different problem than the genericTransformer.

For example If I have 25 cronjob variations I would need 25 subdirectories with a kustomization and a patch file in each and then do something like your suggestion in #38 to generate them all from the same base resource:

kustomizations:
- ./cronjobs/cronjob1
- ./cronjobs/cronjob2
...

The idea for the genericGenerator would be to use 25 genericGenerator's in the base kustomization file instead.

genericGenerator:
  resourcePath: cronjob.yaml
  fieldPathArgs:
  - path: metadata.name
    value: cronjob1
  - path: spec.schedule
    value: "* * * * *"

genericGenerator:
  resourcePath: cronjob.yaml
  fieldPathArgs:
  - path: metadata.name
    value: cronjob2
  - path: spec.schedule
    value: "* * * * *"
...

As for the genericTransformer we can do something with PathConfigs like this:

genericTransformer:
  pathConfigs:
  - path: spec.template.spec.containers.name
    createIfNotPresent: true
    groupVersionKind:
      kind: Deployment
    matches: docker.io/nginx:latest
    value: docker.io/nginx:1.15.0
  - path: spec.jobTemplate.spec.template.spec.containers
    createIfNotPresent: true
    groupVersionKind:
      kind: CronJob
    matches: docker.io/nginx:latest
    value: docker.io/nginx:1.15.0    

[Liujingfang1]

Regardless of the different formats, my understanding based on your examples is that genericGenerator is used to generate new object, while genericTransformer doesn't generate new object, it only updates existing ones.

[sethpollack]

correct.

[sethpollack]

I can work on a PR for the genericTransformer - once Im done working on the secrets PR.

[Liujingfang1]

No rush on that. I need to think more thoroughly on genericTransformer so that it can serve different usage.

[sethpollack]

Ok

[droot]

@sethpollack

For example If I have 25 cronjob variations I would need 25 subdirectories with a kustomization and a patch file in each and then do something like your suggestion in #38 to generate them all from the same base resource:

What is your major concern about the above (current way of doing it) ?

[sethpollack]

@droot I don't think there is a current way - other than duplicating the entire cronjob for each variant of it.

Also I don't think that patches would work anyway since you cant change the name with a patch.

[Liujingfang1]

Let's revisit this request. Recently we added support for multibases with a common base. https://github.com/kubernetes-sigs/kustomize/blob/master/examples/multibases/README.md
So for you use case, you can create multiple cronjobs(as in different variants) from a common base. Then compose those variants together in another overlay.
With this available, I think we don't need genericGenerator. @sethpollack What do you think?

[Liujingfang1]

This genericGenerator idea can be done by JSON patch support #169

[Liujingfang1]

The original request should be able to achieved by multibases support and json patch. Basically, you create a common base for your cronjob. Then create multiple overlays with different prefixes. In those overlays, apply a proper json patch to the cronjob object.

[yubessy]

I've created an example of https://github.com/kubernetes-sigs/kustomize/issues/126#issuecomment-419245703 for future issue visitors like me.
https://github.com/yubessy/example-kustomize-cronjob-multiple-schedule

[abdulsalama]

hmm I actually think that having genericGenerator is still a great feature to add and is really needed. I totally get the point that this can be done with json patch, but it comes with so much unnecessary overhead. Something like genericGenerator would be much cleaner and more intuitive for this kind of need which is very common.