Kustomize: Support variables in ingress resources

[pst]

To allow using the minikube ip command to get the current VMs IP in a development overlay it would be convenient to be able to use variables in ingress resources.

It seems the paths that would need to be supported are (probably incomplete):

{
    GroupVersionKind: &schema.GroupVersionKind{Kind: "Ingress"},
    Path:             []string{"spec", "rules"},
},

{
    GroupVersionKind: &schema.GroupVersionKind{Kind: "Ingress"},
    Path:             []string{"spec", "tls", "hosts"},
},

Example kustomize.yaml

secretGenerator:
- name: minikube
  commands:
    ip: "minikube ip"

vars:
  - name: MINIKUBE_IP
    objref:
      kind: Secret
      name: minikube
      apiVersion: v1
    fieldref:
      fieldpath: data.ip

Example ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: minikube-example
spec:
  tls:
    - hosts:
      - $(MINIKUBE_IP).xip.io
      - api.$(MINIKUBE_IP).xip.io
      - www.$(MINIKUBE_IP).xip.io
  rules:
  - host: api.$(MINIKUBE_IP).xip.io
    http:
      paths:
      - backend:
          serviceName: api
          servicePort: 80
  - host: www.$(MINIKUBE_IP).xip.io
    http:
      paths:
      - backend:
          serviceName: www
          servicePort: 80

Adding the paths to the refVarTransformers path variables seems easy enough. However, the value from fieldpath: data.ip would also need to be base64 decoded. Using the MINIKUBE_IP variable in a container's env currently produces this:

env:
        - name: API_URL
          value: https://api.MTkyLjE2OC4zOS4xNTEK.xip.io

Is this use-case in scope for variables?

[Liujingfang1]

There is a feature request for supporting JSON patch #169, using JSON patch could better solve the problem.

[pst]

I'm not sure I fully understand. I agree that ingress resource patches being verbose is a problem too. My suggestion is to a) allow using a variable inside an ingress resource and b) to base64 decode the value of the variable. From reading #169 it's not immediately clear to me how it would solve my use-case.

My current workaround, maybe this makes it clearer, is to have a placeholder string in my ingress manifest ({{MINIKUBE_IP}}) which I then replace using sed after running kustomize like this:

kustomize build k8s-manifests/overlays/minikube/ \
| sed -e 's/{{MINIKUBE_IP}}/'`minikube ip`'/g' \
| kubectl apply -f -

[monopole]

:1st_place_medal: Using the secret generator's ability to run an arbitrary command to capture build-time data in a secret's field, then use a var binding to inject it into other resources.

I've no objection to allowing use of the downward API in Ingress. Feel free to make a PR for that alone.

As for suppressing the encoding in the secret, expressly to allow kustomize-build-time data in the config - that's contrary to our desire to have a reproducible config stored in version control. The general pattern for injecting build time stuff is to have a sequence like

 kustomize edit set imagetag `script_to_compute_tag`
 # optionally git commit the change to the kustomization
 kustomize build .

If this IP switch is common pattern with ingress perhaps we could make dedicated transformer for it?

Not what you want - but perhaps try https://github.com/kubernetes/minikube/issues/951#issuecomment-381892797 to fix your IP so you can just use a literal IP?

[pst]

Using the secretGenerator was a desperate hack already. I didn't think about kustomize edit but it seems I can't set a variable with set. Allowing the use of variables in an ingress resource would be a small PR only from what I understand.

I understand and agree that my use case goes directly against a reproducible version controlled config. But there are some necessary exceptions already. I merely wanted to discuss if this maybe is another one.

Fixing the minikube IP might be viable for one of my pet projects. But for bigger teams and open source projects it would be great to be able to provide an overlay that adjusts the manifests to the respective local network.

A dedicated generator would probably be the perfect solution. But I'd argue we'd first need to see more people with this use-case before writing and maintaining that code makes sense.

The magic value in the overlay and sed replace workaround for better or worse get's the job done. I'll close this ticket for now. Thanks for your feedback @Liujingfang1 and @monopole.