Kustomize: [Question] Namespace conflict between components

[davidknezic]

Love the idea of components in Kustomize, so I started adapting our many definitions to the structure outlined in the Kustomize componentes guide.

Each of our components usually comes with a namespace definition and a namespace: foo declaration in its kustomization.yaml. This is especially useful in operator components with remote resource files, for which we want to override the default namespace.

Unfortunately, that setup leads to namespace conflicts. You can find a reproduction of my setup below. A kustomize build overlays/local leads to the following error:

Error: accumulating components: accumulateDirectory: "recursed accumulation of path '/Users/davidknezic/repos/github.com/davidknezic/kustomize-ns-per-component/components/db': namespace transformation produces ID conflict: [{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"db\"}}{nsfx:false,beh:unspecified} {\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"db\"}}{nsfx:false,beh:unspecified}]"

components/operator/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

namespace: operator

resources:
- namespace.yaml
- https://k8s.io/examples/application/deployment.yaml

components/operator/namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: operator

components/db/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

namespace: db

resources:
- namespace.yaml
- deployment.yaml

components/db/namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: db

components/db/deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: db
  labels:
    app: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      containers:
      - name: db
        image: hello-world

overlays/local/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

components:
- ../../components/operator
- ../../components/db

Is it possible that components aren't as isolated from each other as I thought or is what I'm seeing unexpected behaviour?

[yanniszark]

cc @apyrgio @ioandr @pgpx

[pgpx]

@davidknezic You are getting the error because of the interaction between the namespace override in the kustomization.yamls and the namespace resources themselves. I'll try to explain (and I was supposed to have provided additional documentation about components - sorry):

If you change the components to normal Kustomizations (and include them as resources in overlays/local/kustomization.yaml then everything works correctly:

• The operator and db are processed independently of each other.
• Each of them defines a namespace resource (db and operator respectively).
• Their kustomization.yaml also specify a namespace: override which overrides the namespace of _all_ resources included by the Kustomization - e.g. the operator and namespace for the operator overlay. Not that the namespace resources are not changed because the override uses the same value, but the operator and db resource do then get the overriding namespace.

Why does using Components change this (and cause the error)? It is due to the differences in the way that Kustomizations and Components are evaluated:

When processing a Kustomization (ignoring child Components for now):

1. A Kustomization defines some resources (in its resources: list) and some transformations to apply to them (in the rest of the file, e.g. the namespace: override). First each child resource is processed completely independently of the other children:
   
   
   
   1. Resource files are just included as-is.
   
   
   2. Nested Kustomizations are processed recursively, and that will result in a set of resource definitions. This processing is not affected by any other Kustomization.
   
   
2. Once all of the child resources are processed (to give a complete set of child output resources), then the transformations in the rest of the Kustomization are applied to _all_ of the child output resources (e.g. to set the namespace on all of them when using a namespace: override).

Components are different! Child components are processed after child resources but before the rest of the transformations. And rather than being completely independent of the other resources and components, when a component is processed it is given the output of all of the previous sibling components and resources, and then its own resources are added and its own transformations are applied to _all_ of them.

So, in your example, when overlays/local/kustomization.yaml is processed:

1. First the operator component will be processed.
   
   
   
   • The processing of the parent Kustomization has not produced any resources (the parent's resources: list is empty), so the only resources to produce are the operator's namespace definition and application deployment.
   
   
   • Its namespace: transformation is then applied to both of these resources.
   
   
2. The db component is then processed.
   
   
   
   • Because it is a Component, it takes as input the resources produced by the operator component.
   
   
   • It then defines its own resources - the db namespace and db deployment, giving a total of 4 resources: operator-namespace, operator-application, db-namespace, db-deployment
   
   
   • The component's transformations are then applied to these 4 resources, which in this case is to set the namespace to db. Unfortunately this will set the namespace of both of the namespace resources to db, which means that we will have duplicate namespace definitions, resulting in your error!
   
   

What should you do?

• In this specific example, I don't think you gain anything by using components, because the db and operator are completely independent of each other anyway, so normal Kustomizations should be sufficient. But I assume that this was a simplified example.
• More generally, I guess don't define namespace: overrides in Components unless you really want to override the namespace of both the children of the component as well as _all_ of the previously-processed resources/components in the parent Kustomization (or parent Component). Note that you could define a namespace override in a Kustomization and then include that Kustomization (e.g. db) as a resource in a parent Component (db-component): the namespace override would only affect the child resources of the db Kustomization (because processing a Kustomisation is independent/self-contained), and would not affect other resources regardless of where the parent component is included.

Sorry - a bit long, but did that make sense? I think I need to use better terms when trying to explain this, and maybe some diagrams!

[davidknezic]

@pgpx, thank you so much for this incredibly valuable explanation!

Indeed, changing my Components to Kustomizations really solves my problem. I don't know why, but I just didn't realize that I can reference _local_ child Kustomizations from my overlay Kustomization. When I stumbled upon Components I imagined that this was what I needed and didn't look further.

Your explanation finally helped me fully grasp the use case for Components! I see now that it's actually described quite clearly in the docs. However, I jumped to the code too quickly and got influenced by the component names ldap, external_db and recaptcha. The sample might have been less confusing for me if the components were called something like with_tls, no_backup or beta.

Thanks again for the clarification!