Application-gateway-kubernetes-ingress: AppGW Ingress Demo not working

[knom]

Hi
I've setup everything (AKS, AAD Pod ID)
After that I'm running the Demo you provided (guestbook + modified ingress file).

When I try to open the URL on the AppGW, I am getting 504 - after a long while..

The GW seems to be configured, I see the Route setup in the AZ Portal..

Hitting the SVC directly or through Node's IP:80 it works..

Looking at the logs of the AppGW Ingress pod, there's no errors..

Any ideas how to debug this?

[asridharan]

@knom did you use the templates provided in the repo to create AKS and Application Gateway v2? Also, are you using Application Gateway v2 or v1? We support only Application Gateway v2 for the ingress controller.

504 error generally means that the Application Gateway is not able to reach the backends (k8s pods). In case of Application Gateway v2 it is imperative that the Application Gateway v2 is in the same vnet as the AKS cluster, else it won't be able to reach the pods. Can you make sure that AKS and Application Gateway v2 share the same VNET (assuming you are using Application Gateway v2).

[knom]

I've not used the Templates..
But I am using AppGateway v2 in the same vnet.
No routes / NSGs or anything else though..

[asridharan]

Is the ingress controller updating the backendpool HTTP setting in AG v2 correctly to the pod's IP address and port (note not the Node's IP address)? Also, please do make sure that pod IP address space and AG address space are independent. Basically just test that you are able to reach the pod subnet space from the AG subnet space.

[knom]

@asridharan: I checked the network route and it seems the POD IP isn't reachable from AppGW.
Also the ROUTE TABLE isn't enabled on the AG subnet..

I'm trying to associate it with the subnet, but getting the following error:

User defined routes are not permitted on Application Gateway subnets when Application Gateways have fast update enabled or have V2 Sku.

[knom]

Update: the POD wasn't reachable because I was using basic networking.
Recreation with advanced networking did the trick!

[asridharan]

Great !!
Thanks @knom for following up on this.

[ddysher]

Hi @asridharan

We are at early stage of investigating azure (aks, to be specific) integration with our infra. It is not strictly required but I'm wondering if it's possible to support application gateway in basic networking?

AFAICT, aws ingress controller supports two modes: instance mode and IP mode, where the former leverages node for traffic routing, and the later requires pod to be routable (probably using aws cni plugin, which comes by default with eks).

[asridharan]

@ddysher the short answer would be that we need advanced networking with AKS to work with the application gateway ingress controller at this point.

The long answer is below:
@ddysher, currently ingress controller only programmes the pods IP address in the Application Gateway and not the node ports IP address. So we need the pods to be routeable directly from the application gateway at this point. To make this possible in Basic Networking you would need to add a UDR to the AG subnet to allow it to route traffic directly to the AKS pods. Since we are using Application Gateway v2 with this ingress controller, unofrtunately, there is no way to add a UDR (user defined route) to the gateway subnet at this point due to which the requirement for using Advanced networking with AKS to make the pods directly routeable the AG.

[ddysher]

@asridharan Thanks for ur prompt response! is it correct to say there is no plan to support basic networking in ingress controller?

btw, i saw user guide from aks, the suggested way to use ingress is to first helm install NGINX ingress controller, is there any plan to make application gateway the default ingress? asking users to install nginx ingress poses unnecessary burdens on user, IMO :)

[asridharan]

@ddysher no plans at the moment to support basic networking with AKS and Application Gateway v2. This limitation exists in Application Gateway v2. This limitation might be removed from AG v2 in the near future but there is nothing allocated on the roadmap yet.

[ddysher]

@asridharan thanks!