Kustomize: Kustomize fails to find object in base modified with nameSuffix

[jessesuen]

Consider the following directory structure:

.
├── aggregate                // contains guestbook-a and guestbook-b as bases.
│   ├── kustomization.yaml   // also applies a patch on guestbook-a
│   └── patch-a.yaml
├── guestbook-a              // contains template as base with nameSuffix: -a
│   └── kustomization.yaml
├── guestbook-b              // contains template as base with nameSuffix: -b
│   └── kustomization.yaml
└── template                 // single resource
    ├── guestbook.yaml
    └── kustomization.yaml

When attempting to build the aggregate, it fails with error:

$ kustomize build aggregate
Error: failed to find an object with ~G_v1_Pod|guestbook-a to apply the patch

• aggregate/kustomization.yaml

bases:
- ../guestbook-a
- ../guestbook-b
patchesStrategicMerge:
- patch-a.yaml

• aggregate/patch-a.yaml

apiVersion: v1
kind: Pod
metadata:
  name: guestbook-a
spec:
  containers:
  - name: nginx
    env:
    - name: FOO
      value: bar

• guestbook-a/kustomization.yaml

bases:
- ../template
nameSuffix: -a

• template/kustomization.yaml

resources:
- guestbook.yaml

I made this available here:
https://github.com/jessesuen/k8s-deployments/tree/master/kustomize-suffix-failed-find-obj

I expect patching to work on objects in the base, even after name transformation using namePrefix or nameSuffix. However, it does not find the object transformed with nameSuffix. Interestingly, if I modify patch-a.yaml to reference the name of the resource without the suffix, it gets the following error:

$ kustomize build aggregate
Error: found multiple objects []resid.ResId{resid.ResId{gvKind:gvk.Gvk{Group:"", Version:"v1", Kind:"Pod"}, name:"guestbook", prefix:"", suffix:"-b", namespace:""}, resid.ResId{gvKind:gvk.Gvk{Group:"", Version:"v1", Kind:"Pod"}, name:"guestbook", prefix:"", suffix:"-a", namespace:""}} targeted by patch resid.ResId{gvKind:gvk.Gvk{Group:"", Version:"v1", Kind:"Pod"}, name:"guestbook", prefix:"", suffix:"", namespace:""} (ambiguous)

This is reproducible with both v2.0.3 and tip of master from today (f9c631e9).

[yujunz]

I was looking for solution for similar scenario, i.e. build multiple persistent volume manifests from the same template base.

IIRC, you should refer the untransformed name in patches, i.e. guestbook instead of guestbook-a.

For the multiple objects error, it seems the same base (the same path) can not be used twice. But you may play a trick by creating symlink into the replicas, e.g. in guestbook-a run ln -s ../template . and use ./template as base. I'm not 100% percent sure it will work though.

Even we get the patches work, there will be further trouble in variable substitution, since it can not be transformed.

My general feeling is that kustomize lacks a feature of making replica of bases. Maybe some new syntax needs to be introduced to support this. It is a quite common scenario to us.

[jessesuen]

IIRC, you should refer the untransformed name in patches, i.e. guestbook instead of guestbook-a.

Yes, I can see kustomize attempting to use untransformed name in the patch. However, referencing the untransformed name in the base is counter intuitive. I expect the patch to be able to reference the object name emitted in the final rendered result of the base. Otherwise it could be a rabbit hole of chasing down the correct original name.

My general feeling is that kustomize lacks a feature of making replica of bases. Maybe some new syntax needs to be introduced to support this. It is a quite common scenario to us.

I agree with this lack of functionality. Essentially we have a template which we want to create multiple instances of, but cannot find a clean way to do this without copying and pasting the entire object and renaming it.

[jessesuen]

I tried to go a different route and use json 6902 for the rename instead of nameSuffix, but I seem to be hitting another error. Filed https://github.com/kubernetes-sigs/kustomize/issues/1083.

[yujunz]

Yes, I can see kustomize attempting to use untransformed name in the patch. However, referencing the untransformed name in the base is counter intuitive. I expect the patch to be able to reference the object name emitted in the _final_ rendered result of the base. Otherwise it could be a rabbit hole of chasing down the correct original name.

Not exactly.

Since currently the same base can be only referred once, it turns out more clear to refer the original name instead of patched one. It would be another story if base replica is supported.

IIUC, the first step in kustomize is building resource map by merging all bases together. Then resolve all the references, apply the patches, transform the names, substitute the variables (not sure about the exact order).

To support base replicas, we kind of need to build bases before merging them, maybe with a new syntax like

replicas:
- overlay1
- overlay2

The overlays will be built and frozen before merging into the aggregated kustomize stack.

[jessesuen]

The reason I feel this is counter-intuitive to refer to the original name, is because it requires understanding of everything that may or may not have happened in the base. That base might have referenced another base, and go several levels of indirection. Hunting down the original resource name seems fragile. In our situation, we even might be leveraging a remote base which is out of our control.

I think the understanding most people have about bases, is that you are applying patches on top of whatever was emitted by that base, after all transformations have been performed. Is my understanding incorrect?

[yujunz]

I think the understanding most people have about bases, is that you are applying patches on top of whatever was emitted by that base, after all transformations have been performed. Is my understanding incorrect?

This is the an intuitive thinking as I did before 😄 but may not be the truth 😞

I haven't dig that far yet. Maybe @Liujingfang1 or @monopole can give a better answer.

[jessesuen]

I think this issue may be related to, if not the same as: https://github.com/kubernetes-sigs/kustomize/issues/316

[Liujingfang1]

When we designed how the patch should work, we didn't include the nameprefix or namesuffix for the target name. That means if you add nameprfix/namesuffix in the patch file, it couldn't find the target. The reason for choosing this behavior is that users don't need to compute or run kustomize build to get the actual target name. Users can use the same resource name from the base. The patching transformer should support it.

@jessesuen @yujunz Based on the examples you provided, it is reasonable that Kustomize is expected to find the correct target. Maybe we can do it as the following

• match object with the same name in the base
• match object with the actual name(with nameprefix and namesuffix)

We say that the target object is found only when there is a unique match.

[yujunz]

Based on the examples you provided, it is reasonable that Kustomize is expected to find the correct target.

Yes, it can.

We say that the target object is found only when there is a unique match.

This is the problem I am trying to solve. It seems a two-pass build might be the right way to do it.

With @jessesuen 's example

.PHONY: guestbooks.yaml

guestbooks.yaml:
    echo "DO NOT EDIT - generated by make" > guestbooks.yaml
    kustomize build guestbook-a.yaml >> guestbooks.yaml
    kustomize build guestbook-b.yaml >> guestbooks.yaml

Is there something like pre-build hook we can leverage in kustomize? @Liujingfang1

[erick-thompson]

I'd rather avoid a two-pass build. I like the approach of matching to both base name and to name prefix/suffix. Is this something that is on the roadmap?

[botwhytho]

I can open different ticket if scope of fix would vary, but seeing similar unintuitive behavior even if I use a json patch in a base to patch a Deployment name.

In an inheritance scenario like the one below, find it logically incorrect to have to refer to a Deployment resource by its base name in a strategic merge patch at a child environment overlay level, when I already patched its name in a parent base. Kustomize will output the output I want but even after understanding current functionality, having to document and explain the patching logic to others is not a pleasant experience.

Example Inheritance

Remote Base > Local Base > Service Overlays (service specific changes) > Environmental Overlays

The suggestions in https://github.com/kubernetes-sigs/kustomize/issues/727 don't seem to actually work in my scenario. I originally tried to use nameSuffix in the way this ticket is suggesting with no luck, and ideally I prefer outright patching of resource names at times, even if more verbose.

As mentioned already, working with remote bases starts getting rather dicey as you'll need to know the names of root resources that you may or may not manage. This also prevents scenarios like having 1 json patch that is shared among resources (where use of yaml anchors would be super useful).

Again, can definitely open new ticket if need be, modifying resource names with nameSuffix is not my ideal end state, but if a fix lands only with name{Suffix,Prefix} it'd be better than current state.

[botwhytho]

• match object with the actual name(with nameprefix and namesuffix)

@Liujingfang1 could this name matching apply more generally to actual name (modified with a patch on a parent base)?

[erick-thompson]

@botwhytho If you could open a ticket, that would be awesome. I'm running into some serious headaches when I have a namePrefix or nameSuffix before the overlay. Having kustomize match on the base name, and if not found, match on name{Suffix|Prefix} would fix my issue.

The scenario is I have a remote (git) base that contains a base ElasticSearch setup, and I pull this in to a base folder with a namePrefix (to ensure a unique service name). The overlay then does things like configure hostname.

[erick-thompson]

@monopole Do you know when #1154 will be released?

[stas2k]

This makes kustomize unsuitable for my use case - reusing base layers to describe similar app configuration.
Is it possible to put a notice on kubectl docs about this issue?

[benjamin-bergia]

I am hitting the same issue here and it's becoming borderline unusable. I have the following scenario:

env/
  base/
    kustomization.yml //resources: app/overlays/web, app/overlays/worker
    app/
      base/
        deployment.yml
        kustomization.yml //resources: deployment.yml
      overlays/
        web/
          kustomization.yml //base: ../base
        worker/
          kustomization.yml //base: ../base
  overlays/
    prod/
      kustomization.yml //base: ../base
    qa/
      kustomization.yml //base: ../base

My base defines the deployment, web/worker update the command/arguments and add a suffix to the name. In prod/qa, i define the namespace and I would like to define the replica count. Unfortunately, it is not possible this way. When using replicas in the qa kustomization, the deployment names don't contain the suffixes yet. The replicas count is related to the type (worker or web) but also the environment (qa or prod).

[Liujingfang1]

This has been fixed by #1154. Please check out version 3.0.0-pre.

[monopole]

Calling this fixed by #1183, which is in v2.1.0.

Added a test to cover in #1278

How this should work exactly is open for debate, but the behavior is

• a patch is defined, and must have a unique target (NsGVKN).
• try to find an exact match by the original name, e.g. ConfigMap/foo
  (frees user from having to do a mental kustomization of the base)
• failing that, try to find by the anticipated final name, e.g. ConfigMap/prefix-foo

a feature allowing a patch to have multiple targets is coming soon.