Kustomize: Overlay configs for custom generators do not override base configs

[snarlysodboxer]

I'm using Kustomize 3.2.0.

It seems to be impossible to use a base/overlay setup for custom generator configs. Generator configs in overlays do not override generator configs in base, at least not prior to running the custom plugin. E.G.

Base (for a staging environment):

kustomization.yaml:

generators:
- mySecretGetter.yaml

mySecretGetter.yaml:

apiVersion: crd.myteam.com/v1
kind: SecretGetter
metadata:
  name: my-name
secretsProjectID: 1234
...

Overlay (for a production environment):

kustomization.yaml:

resources:
- ../../base

generators:
- mySecretGetter.yaml

mySecretGetter.yaml:

apiVersion: crd.myteam.com/v1
kind: SecretGetter
metadata:
  name: my-name
secretsProjectID: 5678

What I did:

• Tried to kustomize build the overlay.

What happened:

• The plugin tries to get secret values using secretsProjectID: 1234.
• This happens during the resource inclusion of ../../base.
• My plugin fails here because the production credentials are not authorized to the staging secrets project 1234.

What I expected to happen:

• The plugin tries to get secret values using secretsProjectID: 5678.

Additional Notes

It's possible to move mySecretGetter.yaml into the resources section in the base, and patch it in the patches section in the overlay before using it as a generator config, however this produces objects of kind SecretGetter in the build output, as well as makes the base an incomplete configuration on it's own.

The work-around I'm currently using is to use separate overlays, one for staging and one for production, and a Kustomize structure like the following. This is not ideal because the base is again not a complete configuration on it's own, and because it requires nearly identical complete copies of mySecretGetter.yaml, which deviates from Kustomize's base/overlay strategy.

Staging Overlay (work-around):

resources:
- ../../base
- ../../secrets/staging

Production Overlay (work-around):

resources:
- ../../base
- ../../secrets/production

kustomization.yaml in secrets/staging and secrets/production:

generators:
- mySecretGetter.yaml

[Liujingfang1]

In the overlay, instead of adding mySecretGetter.yaml as generators, add it as a patch.

resources:
- ../../base

patchesStrategicMerge:
- mySecretGetter.yaml

[snarlysodboxer]

@Liujingfang1 Thank you. I tried that. The problem is it first runs the plugin with secretsProjectID: 1234, even though the patch later tries to change it to secretsProjectID: 5678. This fails because where my production overlay is being run, it's credentials are not authenticated to the staging project ID 1234, only the production project ID 5678. Then, since the base didn't produce the object from mySecretGetter.yaml, the patch also fails with failed to find unique target.

[snarlysodboxer]

Is there any plan to support not running plugins until after their configs have been patched by overlays?

[tehmoon]

I ended up in the same situation and try to play with the suggested answer with patchesStrategicMerge in the overlay.
I tried to add a flag enable: true in the base/mySecretGetter.yaml that will output the current templated configuration so it could be merged to your staging/mySecretGetter.yaml. The the goal would to have enable: true in the overlay so the plugin runs correctly (there is additional logic in the plugin).

That is patching the base/mySecretGetter.yaml correctly, but not executes the generator after :/ If there is a way to execute the generator generated by by the output of the patchesStrategicMerge that would solve my issue.

[snarlysodboxer]

@Liujingfang1 Additionally, unless you add mySecretGetter.yaml to both the resources and the generators in your base, (_and_ are ok with your plugin config being output by kustomize build) you cannot have mySecretGetter.yaml in patchesStrategicMerge section in your overlay, because kustomize will fail with failed to find unique target for patch crd.myteam.com_v1_SecretGetter|example-login.

This is quite the problem as various workarounds are all also problematic, and it seems like a common scenario for certain types of custom plugins. We need a way to patch plugin configs _before_ the plugin gets run against them.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.