Kustomize: image name/tag cannot be set from base

[donbowman]

Consider this example. In 'parent' I have a deployment using an image. In overlay i have base ../parent and base ../common. I wish to set the image tag in common. It does not work.
I can set it from overlay or parent but not common. An example is attached.

My desire is to have one spot to change the image tag for a set of related things. Instead I am forced to cut and paste the same text over and over in the overlays.

├── common
│   └── kustomization.yaml
├── overlay
│   └── kustomization.yaml
└── parent
    ├── dep.yaml
    └── kustomization.yaml

image-tag-hierarchy.tar.gz

[Liujingfang1]

images are only effective to current kustomization and are not visible to its overlays.

[donbowman]

but why, is this by design? or by chance?

[Liujingfang1]

This is by design. Similar to namePrefix or commonAnnotations, they only set or modify the resources manged by the current kustomization.

[donbowman]

ok.... but is this important?
its making quite a bit more work and harder to maintain.

[yujunz]

It sounds reasonable to me to limit the transform within each kustomization. One benefit would be a deterministic build. Another would be flexibility to allow different transform in sibling overlays.

[gautaz]

@yujunz I don't see how this limitation ensures a more deterministic build.

As soon as you are relying on base files, any modification to these base files impacts the kustomization using them, be it resources or images seems like the same to me.

Being able to specify images in base files also does not cripple flexibility for overlays, you can still modify again images in each overlay kustomization file.

From what I understand, it seems that this feature would increase even more the ability to factor kustomization code.

Would you mind detailing a bit more why this feature would impeach deterministic build and flexibility (perhaps through examples)?

[nickatsegment]

Are contributors still against this on principle? When I found this out, it was really counterintuitive. The limitation is only documented in this bug AFAICT, not docs: https://github.com/kubernetes-sigs/kustomize/blob/c094780aaedd241fedd34d73d409d1699488f167/docs/fields.md#images.

Similar to namePrefix or commonAnnotations, they only set or modify the resources manged by the current kustomization.

commonAnnotations seems to work fine in a base for me.

[yujunz]

Would you mind detailing a bit more why this feature would impeach deterministic build and flexibility (perhaps through examples)?

@gautaz Just created an example of different tags in one kustomize build.

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: X-dev
  namespace: X
spec:
  replicas: 1
  template:
    spec:
      containers:
      - image: cr.agilicus.com/group/image:dev
        name: X
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: X-stage
  namespace: X
spec:
  replicas: 1
  template:
    spec:
      containers:
      - image: cr.agilicus.com/group/image:stage
        name: X

See https://github.com/yujunz/image-tag-hierarchy for the hierarchy.

[gautaz]

Hello @yujunz,

OK, I see your point now. For the example to be more complete, a new deployment using the image cr.agilicus.com/group/image should be declared in the overlay and the question that would arise would be which tag will use this deployment?

Still this could be rendered deterministic by simple rules like "last one wins", which would mean that the stage tag would be applied to this overlay declared deployment.

I think we have in mind two different use of bases.

Your example shows this type of dependency:

• O -> B1
• O -> B2

My intent (and perhaps what is needed by @donbowman) is more to obtain a DRY approach for this type of dependency:

• O1 -> B
• O2 -> B

Both types of use can occur and supporting things like "last one wins" would still make the result deterministic. However I agree that implementing this is perhaps not easy.

[Oduig]

I have a use case for this. Let's say we have the following GitOps setup.

• base
  
  
  
  • deployment.yaml
  
  
  • kustomization.yaml
  
  
• overlay
  
  
  
  • dev
    
    
    
    
    
    • configmap.yaml
    
    
    
    • kustomization.yaml
    
    
    
  
  
  • tst
    
    
    
    
    
    • configmap.yaml
    
    
    
    • kustomization.yaml
    
    
    
  
  
  • acc
    
    
    
    
    
    • configmap.yaml
    
    
    
    • kustomization.yaml
    
    
    
  
  
  • prd
    
    
    
    
    
    • configmap.yaml
    
    
    
    • kustomization.yaml
    
    
    
  
  

I would like my deployment to defined in the base, so we set the image in base/kustomization.yaml. This works for tst, acc and prd, but for dev we might want something different, e.g. a snapshot build of the latest commit or an image with additional debug tooling. It would be useful to override the image of base/kustomization.yaml in overlay/dev/kustomization.yaml.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[Oduig]

/remove-lifecycle rotten

[robinbraemer]

I guess to have a workaround to achieve what @Oduig said one must omit the images: in the base kustomization.yaml and declare it in each overlay so there the base has no image to to override.

Is this still being worked on to override the base transformers/images from an overlay,
since doing it my way is not great?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[adoyle-h]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[Oduig]

/remove-lifecycle stale

[Oduig]

Any news on this, will this feature be considered for implementation?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[Oduig]

/remove-lifecycle stale

[Oduig]

It's been over a year since I posted my use case for this feature. Can we get a word on whether or this is desirable to be implemented and can we stop this bot from marking it as stale for the 10th time?

[Shell32-Natsu]

@donbowman IMO, your use case is counter to the kustomize design. Each layer, or kustomization is self-contained. ../common shouldn't have any knowledge of its sibling ../base (because they are organized by overlay and don't have a solid logic relationship).

The use case from @Oduig makes more sense. However I cannot see what is preventing you from modifying the images name and tag in overlay. Image transformer should be able to override the value from base.

[donbowman]

i ended up writing a plugin transformer https://github.com/Agilicus/kustomize-plugins/tree/master/agilicus/v1/imagetransformer

otherwise its too complex to get it right.

[Oduig]

It is possible to achieve the same effect by adding the image to each overlay. It would just be more DRY if, when we have a number of overlays share the same image, we could specify the image in base and override it only when necessary. After all, not repeating shared configuration is sort of the point of Kustomize.

I will attempt to explain our situation for some background information.
In our project, we have a gitops setup with a branch for each environment. The dev overlay on the dev branch is automatically deployed with ArgoCD, as is the tst overlay on the tst branch, etc. When we finish a feature, we update all the kustomize overlays on branch dev to reflect the new version number tag. This means the production config is already written as part of the feature. When the time comes to release, we merge branch dev (with the updated overlays) into branch tst, later acc, then prd. By doing these merges we can ensure that we have the exact same image tag that is deployed to tst, also deployed to acc and prd.

That said, the loophole in this process is that someone might update a version number only in the tst overlay, and forget to do it for acc and prd. So when we merge the tst branch into acc, the acc overlay still has the old tag and no deployment is done by ArgoCD. We could avoid that scenario with an image tag specified in the base: anyone who would update the tag for one deployment has done so for tst, acc and prd at the same time. The dev overlay stays the same here because it always points to :latest.

[Shell32-Natsu]

@Oduig Thanks for the description. I have 2 questions:

1. _So when we merge the tst branch into acc, the acc overlay still has the old tag_
   If you update the tag in tst but not in acc. How do you merge to acc without conflict or override? I suppose tst should be ahead of acc.

2. _We could avoid that scenario with an image tag specified in the base: anyone who would update the tag for one deployment has done so for tst, acc and prd at the same time._
   You can put the image in the base and it will be applied to all overlays. Meanwhile set the image tag only in dev. Doesn't this achieve the goal?

[Oduig]

@Shell32-Natsu Thank you for the patience and taking a look at this :+1:

It has been a while since I wrote this code. I thought I had remembered the specifics, but in fact my memory failed me!
What you describe (point 2) is true: you can override the image in one overlay and use the base image in the other overlays. The thing that does not work, is overriding the image and newTag in the kustomization.yaml.

Edit: It turns out that newTag is working as intended. Apparently, you cannot override an image definition. Instead, you can chain them and use the newName from the base as a name in the overlay, that way it will update the tag. My question is accurately covered by this issue here: https://github.com/kubernetes-sigs/kustomize/issues/1040.

It looks like I misunderstood this when I made my original comment a year ago! For my POV this can be closed.