Kind: Ingress on different Ports Question

[shashank-g172]

Hi!

First, Kind is awesome and thank you for creating and maintaining it. I have a couple of questions that sort of follow along the lines of this post: https://github.com/kubernetes-sigs/kind/issues/1411

I'm working on a VM which already has a web server running on Port 80, so I followed through the Kind ingress documentation , and I changed the port that the cluster's ingress is supposed to be on from 80 to 8080, and from 443 to 9000. The files that I have used are below.

I saw that we're trying to folllow the 80 -> 80 -> 80 rule all the way through, but for situations when port 80's unavailable for whatever reason, is it fair to say that the following steps below should be the workaround?

After patching through the deployment, I left the example foo-bar service untouched since they're on port 5678, but when I tried to curl it on the expected host and port, I get back a response saying that the service sent an empty response.

Here are my files

kind create cluster with config:
```kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:

• role: control-plane
• role: worker
• role: worker
  kubeadmConfigPatches:
  
  
  
  • |
    
    kind: InitConfiguration
    
    nodeRegistration:
    
    kubeletExtraArgs:
    
    node-labels: "ingress-ready=true"
    
    authorization-mode: "AlwaysAllow"
    
    extraPortMappings:
  
  
  • containerPort: 80
    
    hostPort: 8080
    
    protocol: TCP
  
  
  • containerPort: 443
    
    hostPort: 9000
    
    protocol: TCP
  
  

Then:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml

kubectl patch deployments -n ingress-nginx nginx-ingress-controller -p '{"spec":{"template":{"spec":{"containers":[{"name":"nginx-ingress-controller","ports":[{"containerPort":80,"hostPort":8080},{"containerPort":443,"hostPort":9000}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

Then I tried using the same example:

kind: Pod
apiVersion: v1
metadata:
name: foo-app
labels:
app: foo
spec:
containers:

• name: foo-app
  image: hashicorp/http-echo
  args:

- "-text=foo"

kind: Service
apiVersion: v1
metadata:
name: foo-service
spec:
selector:
app: foo
ports:
# Default port used by the image

- port: 5678

kind: Pod
apiVersion: v1
metadata:
name: bar-app
labels:
app: bar
spec:
containers:

• name: bar-app
  image: hashicorp/http-echo
  args:

- "-text=bar"

kind: Service
apiVersion: v1
metadata:
name: bar-service
spec:
selector:
app: bar
ports:
# Default port used by the image

- port: 5678

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:

• http:
  paths:
  
  
  
  • path: /foo
    
    backend:
    
    serviceName: foo-service
    
    servicePort: 5678
  
  
  • path: /bar
    
    backend:
    
    serviceName: bar-service
    
    servicePort: 5678
  
  

I even see that everything's created just as expected:

kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
bar-app 1/1 Running 0 9s 10.244.0.8 kind-control-plane
foo-app 1/1 Running 0 9s 10.244.0.7 kind-control-plane

kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
example-ingress * 80 31s

curl localhost:8080/foo
curl: (52) Empty reply from server

```
I apologize for the simplistic nature of the question but changing the hostPort shouldn't cause the example to break right?

Several hours and countless experiments with NodePort and reading through this which @BenTheElder has referenced:http://alesnosek.com/blog/2017/02/14/accessing-kubernetes-pods-from-outside-of-the-cluster/ - I'm still scratching my head.

I'm using tcsh for reference on a Debian 10 machine.

[shashank-g172]

Nope, I didn't deploy the contour components. Ok, I feel sheepish now.

It's working as expected. My bad. Thanks anyway!

[shashank-g172]

I feel like I'm thinking in real-time, sorry for the spam. I shouldn't need to have to set up nginx AND contour for my app to be available - if it's either or - my nginx configuration is still missing something.

Any thoughts?

[amwat]

I was able to make this work with a couple of changes:

1. you are missing kind: Cluster in the create config.
2. since you are exposing a worker node instead of a control plane node you should be using JoinConfiguration instead of InitConfiguration.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
        authorization-mode: "AlwaysAllow"
  extraPortMappings:
  - containerPort: 80
    hostPort: 8080
    protocol: TCP
  - containerPort: 443
    hostPort: 9000
    protocol: TCP

1. you shouldn't need to modify the patch you can use the patch as is. The "hostPort" of the patch is not the port on the host, it's the port on the "node" container.

kubectl patch deployments -n ingress-nginx nginx-ingress-controller -p '{"spec":{"template":{"spec":{"containers":[{"name":"nginx-ingress-controller","ports":[{"containerPort":80,"hostPort":80},{"containerPort":443,"hostPort":443}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

[electrofelix]

Does the ingress.kubernetes.io/rewrite-target: / handle changing the ports in the likes of redirect urls?

[aledbf]

Does the ingress.kubernetes.io/rewrite-target: / handle changing the ports in the likes of redirect urls?

No. Only URLs

[shashank-g172]

That worked! I missed adding the kind:Cluster in the config - copy-paste error my bad.

you shouldn't need to modify the patch you can use the patch as-is. The "hostPort" of the patch is not the port on the host, it's the port on the "node" container. - Yup, that's what tripped me up. I kept re-reading the docs to try to figure out why we're doing this - here's the only doc about it though:

Apply kind specific patches to forward the hostPorts to the ingress controller, set taint tolerations and schedule it to the custom labelled node.

Thanks a ton @amwat! You're a hero!