ingress-nginx LoadBalancer with multiple namespaces

[mattcopas]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/.):

Yes.

Seen https://github.com/kubernetes/kubernetes/issues/17088
https://github.com/kubernetes/ingress-nginx/issues/1225
https://github.com/kubernetes/ingress-nginx/issues/1976

but none of those helped.

What keywords did you search in NGINX Ingress controller issues before filing this one? (If you have found any duplicates, you should instead reply there.):

"multiple namespaces", "namespaces", "namespace"

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Neither

NGINX Ingress controller version:
0.18.0 (controller image is quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0)

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335",GitTreeState:"clean", BuildDate:"2018-05-12T04:12:12Z", GoVersion:"go1.9.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.4-gke.3", GitCommit:"eb2e43842aaa21d6f0bb65d6adf5a84bbdc62eaf", GitTreeState:"clean", BuildDate:"2018-08-04T03:45:50Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

Environment: GKE

• Cloud provider or hardware configuration: GKE

I recently started using Ingress Nginx on a cluster with a project deployed in a single namespace and it works fine. I deployed the third-party ingress nginx stuff from https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml (using GCE, following the getting started guide)



However, I’m not sure what to do now that I have multiple namespaces, as ingress-nginx deploys just 1 load balancer.

I need a way of directing traffic to the correct namespace

I see 2 solutions to this:

1. A loadbalancer per project namespace - this doesn’t really make sense as the loadbalancer (ingress-ngnix) is in the ingress-nginx namespace. Also the ingress-nginx LoadBalancer comes from https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml so I'm guessing changing this is not the way to go.

2. Telling the ingress-nginx loadbalancer how to deal with namespaces…eg loadbalancer_ip/namespace-1/web goes to the web app in namespace-1, and ditto for namespace-2

3. Some other way I haven't thought of?

The ingress-nginx loadalancer urls still work (ie if I go to loadbalancer_ip/web I get directed to the web app, but I don’t know which namespace the app is coming from or how I’d control the namespace)

Essentially, I have 2 copies of a project, in 2 namespaces, and need to know how to control which namespace the loadbalancer/ingress-controller will send traffic to.

I hope my question is clear!

Thanks for any help

[aledbf]

@mattcopas why you want to use two loadbalancers? With one ingress-nginx you can have two hostnames, like prod.xxxx.com and dev.xxxx.com in two different Ingress rules, one in each namespace where your app is defined

[aledbf]

Telling the ingress-nginx loadbalancer how to deal with namespaces…eg loadbalancer_ip/namespace-1/web goes to the web app in namespace-1, and ditto for namespace-2

You don't need to "tell" the ingress-nginx loadbalancer how to deal with namespaces. Just defined the Ingress rule in each namespace and by default the ingress controller will detect those definitions

[mattcopas]

@aledbf But the ingress rules are just paths (this might be my issue)

e.g. my ingress for both namespaces looks something like:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-project-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

which (to me at least) means load_balancer_ip/ goes to the web svc in the namespace that the ingress is defined in

Or are you saying I can add hostnames to that manifest?

It might be worth mentioning that currently I have no DNS set up for this project (although it is something we'll be doing in the coming days)

Thank you very much

[aledbf]

But the ingress rules are just paths (this might be my issue)

Exactly

Or are you saying I can add hostnames to that manifest?

You should do that.

It might be worth mentioning that currently I have no DNS set up for this project (although it is something we'll be doing in the coming days)

You can add that later. Using kubectl get ingress -o wide you should see the IP address of the load balancer assigned. You can just use that information to update your local etc host file to test all this

[aledbf]

Closing. Please use the #ingress-nginx slack channel to ask questions about the ingress controller

[kfox1111]

or play with http://xip.io/
Great for testing this sort of thing. :)

[aledbf]

http://xip.io/

That works too :) (until you need ssl)

[mbalanced]

@aledbf is it possible to create one ingress to handle multiple subdomains with different service and each service in a separate namespace?

[aledbf]

@mbalanced you need to create the ingress where the service is defined. Just define multiple ingress rules

[scparker75]

@aledbf So this can be done under one ingress you just need to define rules?

[aledbf]

@scparker75 no if the services are located in different namespaces

"one ingress" you mean one Ingress definition or one ingress controller? (naming is hard :wink: )

[scparker75]

@aledbf One controller

[aledbf]

One controller

Yes, just configure the Ingress definition in the same namespace than the service

[mbalanced]

@aledbf I have 3 subdomains, each domain serves a different app, each app "deployment, service, ingress" located in a different namespace. that's mean we have 3 ingresses across 3 different namespaces. the question is that possible to use 1 ingress across 3 of namespaces instead of 3?

[aledbf]

@mbalanced no

[mbalanced]

@aledbf do you have any a workaround to solve this?

[aledbf]

do you have any a workaround to solve this?

No, you need to use different ingress definitions in each namespace

[scparker75]

If we put it under one namespace?

[znat]

The only problem is that every time you create a namespace, all nginx pods are restarted, closing all current connections with all namespaces.

[linuxidefix]

A good explanation on multiple namespaces with subdomains by environment: https://cloud.ibm.com/docs/containers?topic=containers-ingress#multi-ns

[Floeig]

If you use traefik instead it will work