Terraform-provider-kubernetes: Feature request: Ingress resource

[holoGDM]

Hello

There is lack of ingress resources support in Terraform. Please can you add it?

Regards
holoGDM

[radeksimko]

Hi @holoGDM
unfortunately ingress is currently in v1beta phase: https://kubernetes.io/docs/resources-reference/v1.6/#ingress-v1beta1-extensions

As mentioned in the Readme, for reasons discussed in https://github.com/terraform-providers/terraform-provider-kubernetes/pull/1#issuecomment-307940033 there are currently no plans on supporting any API that's marked as alpha, beta or generally anything that's not stable/v1.

Thanks for understanding.

[sl1pm4t]

Hi @holoGDM - I've just added support for the Ingress resource over in my fork:
https://github.com/sl1pm4t/terraform-provider-kubernetes/releases/tag/v1.0.2-custom

See example usage here:
https://github.com/sl1pm4t/terraform-provider-kubernetes/tree/custom/_examples/ingress

Obviously use with caution because this is not official and has no guarantees for backwards compatibility going forward.

[aeneasr]

Since this isn't currently possible, how would I go about adding an HTTPS proxy in front of the LoadBalancer using the GCP provider?

As far as I understand, type="LoadBalancer" creates a GCP Backend Service with a frontend that listens on randomly assigned IP Address (frontend) and the a (virtual?) instance group pointing to GKE (backend). That works nicely, however, it's not possible to link a GCP HTTPS Proxy with that IP address assigned to the frontend, as I'm only able to connect that proxy with a GCP Backend Service, which requires an instance group. Since there isn't a way to query for that created backend (load_balancer_ingress only gives the IP address) it's basically impossible to do this.

I would appreciate any help/pointers/RTFM regarding that as I'm stuck.

[tsirlucas]

Any updates on this one? I'm going through the same problem.

@arekkas Did you find a workaround for your problem?

[iorlas]

Same here, waiting, crying

[aeneasr]

I gave up on it and just added HTTPS to my service.

[aeneasr]

Check this: https://github.com/GoogleCloudPlatform/terraform-google-lb-http

[Radik2]

Hello all,
Any updates on this one? I have same problem but with OpenStack :( .Are there any plans to add Ingress resource soon ?

[Radik2]

Hello @sl1pm4t ,

Has this https://github.com/sl1pm4t/terraform-provider-kubernetes/blob/custom/_examples/ingress/main.tf ever worked or it work only with v1beta ? I have Terraform v0.11.10 with provider.kubernetes v1.3.0.
When I run "kubernetes_ingress" part of the code it says : "Provider doesn't support resource: kubernetes_ingress"

[zero-below]

@Radik2,

The blurb you referenced works with a provider built from sl1pm14t's provider fork, not the official fork.

As a side note, I see up above in comments in this thread, people trying to get ssl to work. Here's how to do it.

    annotations {
      "ingress.gcp.kubernetes.io/pre-shared-cert" = "${ var.ssl_cert_name }"
      "kubernetes.io/ingress.allow-http" = "false"
      "kubernetes.io/ingress.global-static-ip-name" = "${ google_compute_global_address.lb-address.name }"
    }

[acobaugh]

I normally voice support for an issue with a thumbs up to avoid spam, but this is getting somewhat ridiculous. Any ETA on when ingress will be supported? I've been using @sl1pm4t's fork with great success, but the setup to use that provider instead of simply being able to do terraform init is quite cumbersome when on a team...

I really want to use terraform to manage kubernetes, as I and others believe it is far superior to helm charts (templating out structured text?) or writing out endless yaml files. However, lack of support for even Stable features is a potential show-stopper for us.

[sarneaud]

However, lack of support for even Stable features is a potential show-stopper for us.

The Ingress resource is still in beta:
https://kubernetes.io/docs/concepts/services-networking/ingress/#prerequisites

[aeneasr]

I agree with @acobaugh 's sentiment. And by the way, the Google Provider is exposing a beta provider which leaves it to the implementer's discretion as to whether use it or not. I agree that simple use cases as setting up SSL for inbound traffic (via Ingress), which is supported on major platforms for over a year (years?) really puts a limit to the usefulness of the k8s provider, significantly increases developer frustration (as seen on this thread) and doesn't make a lot of sense as we're seeing other providers supporting beta features. On top of that, the community is apparently willing to provide enhancements and code changes that support these features, but it's really up to the maintainers to finally take the necessary step.

LETS FINALLY DO SOMETHING ABOUT IT

[cupojoe]

I'm not sure what the deterrent to implementing Google's beta API is. Google is famous for leaving very stable products in beta for years.
I think implementing something like @sl1pm4t solution and flagging/warning the docs with a beta status would be sufficient to make the community happy and the k8 provider much more useful.
To get around this I've been using a sh script to create the ingress, but it's ugly to say the least.
Would love to hear from the maintainers why such a hard stance on the beta status from Google.
This is my ingress script in case it helps someone:

## Get creds from the cluster. This can be done by passing a KUBECONFIG env variable instead, setting it in terraform
echo "Updating cluster credentials"
gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE
echo "Creating ingress resource"
# Dynamically replace the name of the static ip resource in the yaml file
INGRESS_YAML=`cat "ingress/ingress.yaml" | sed "s/{{IP_NAME}}/$STATIC_IP_NAME/g" | sed "s/{{INGRESS_NAME}}/$INGRESS_NAME/g"`
echo "$INGRESS_YAML" | kubectl apply -f -
external_ip=$(kubectl get ingress my-ingress-name --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}")
time_spent=0
time_out=300
while [ -z "$external_ip" ] && [ "$time_spent" -lt "$time_out" ]; do
    external_ip=$(kubectl get ingress my-ingress-name --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}")
    time_spent=$(($time_spent+5))
    sleep 5
done

if [ -z "$external_ip" ]
then
    echo "Time out. Unable to create ingress. Try running terraform apply again."
    exit 1
else
    echo "Load balancer ready at: $external_ip"
fi

I call this from a null local exec resource like this:

resource "null_resource" "my-ingress" {
  # Required so this triggers every time. It won't affect the ingress if nothing has changed
  triggers {
    key = "${uuid()}"
  }
  provisioner "local-exec" {
    command = ". ingress/create-ingress.sh"
    environment {
      SERVICE_NAME = "${var.service_name}"
      CLUSTER_NAME = "${var.cluster_name}"
      STATIC_IP_NAME = "${var.static_ip_name}"
      INGRESS_NAME = "${var.ingress_name}"
      ZONE = "${var.zone}"
      REGION = "${var.region}"
    }
  }
}

[Radik2]

Hello again guys :) , Hello @sl1pm4t ,

Is there "secretName" as a rule in or something else we can set in fork Ingress resource ? I have tried to set "secretName" , "secret_name" or "secretname" but Terraform Fork does know about that...Like this for example

rule {
host = "myminikube.info"
secret_name = "name_of_secret" --> "invalid or unknown key: secret_name"

  http {
    path {
      path_regex = "/"

      backend {
        service_name = "echoserver"
        service_port = 8080
      }
    }
  }

}

My idea is to secure the Ingress resource ...Is there any other way to secure it ?
Any help will be appreciated :)

[Radik2]

Hello :) ,

Could somebody help me ?I think it does not make sense to create ingress if it is not secured...
Thank you!

[sl1pm4t]

Hi @Radik2
The secret_name attribute needs to be nested instead a tls block like:

...
  spec {
    tls {
      secret_name = "tls-secret"
    }
...

[Radik2]

Hello @sl1pm4t and thank you very much for your answer . It works but something else is worrying me :
I set following structure
spec {
tls {
hosts = "example.com"

  secret_name = "tls-secret"
}

}

And terraform said "hosts: should be a list" and then I set to :

spec {
tls {
hosts = ["example.com"]

  secret_name = "tls-secret"
}

}

Which was accepted as syntax but in Kubernetes UI in Ingress yaml file I see following configuration uploaded:

"hosts": [
"example.com"
],
And those "[" brackets I think should not be there...? Might be I do not understand something...

[Radik2]

Hello @sl1pm4t and thank you very much for your answer . It works but something else is worrying me :
I set following structure
spec {
tls {
hosts = "example.com"

  secret_name = "tls-secret"
}

}

And terraform said "hosts: should be a list" and then I set to :

spec {
tls {
hosts = ["example.com"]

  secret_name = "tls-secret"
}

}

Which was accepted as syntax but in Kubernetes UI in Ingress yaml file I see following configuration uploaded:

"hosts": [
"example.com"
],
And those "[" brackets I think should not be there...? Might be I do not understand something...

Please ignore above message . I have tested with kubectl and yaml. file and it work like that so no worries

[Radik2]

Hello guys,

Let me ask you if you have ever had a problem with the "tfstate" file while initializing the backend which in my case is Nexus or Artifactory :
Initializing the backend...

Successfully configured the backend "artifactory"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: Get hostNameOfRemoteArtifactory: dial tcp someipaddress:port: getsockopt: no route to host

I have tested with official release for Kubernetes here https://www.terraform.io/docs/providers/kubernetes/guides/getting-started.html and it works :)

Thank you in advance

[tdmalone]

Hi @Radik2, this issue is about a feature request for the Ingress resource. You'd be best posting a new issue to request help on something unreleated to this issue :)

[tburow]

Updates on this feature request? seems this is somewhat a core feature that should be added. with out it, I'm finding this provider useless like getting to the 1 yard line and going home..

[tburow]

Here is an example for a workaround, but still would like this resolved. I hope this snippet helps others.

data "template_file" "ingress" {
  template = "${file("templates/ingress.yaml.tpl")}"
  vars {
    INGRESS_NAME = "<value>"
    HOSTNAME     ="<value>"
    PATH         = "<value>"
    SERVICENAME  = "<value>"
    SERVICEPORT  = "<value>"
  }
}

resource "null_resource" "ingress" {
  triggers = {
    manifest_sha1 = "${sha1("${data.template_file.ingress.rendered}")}"
  }

  provisioner "local-exec" {
    command = "kubectl apply -f -<<EOF\n${data.template_file.ingress.rendered}\nEOF"
  }
}

ingress.yaml.tpl
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ${INGRESS_NAME}
spec:
  rules:
  - host: ${HOSTNAME}
    http:
      paths:
      - path: ${PATH}
        backend:
          serviceName: ${SERVICENAME}
          servicePort: ${SERVICEPORT}

[roidelapluie]

I agree that ingress is ridiculously missing in this provider

[tburow]

bump - Im pushing on both repos/project to resolve this. seems easy enough since the code was already written and in a PR.

[eveenendaal]

Maybe the feature could be added to a kubernetes-beta provider like the google-beta provider? https://www.terraform.io/docs/providers/google/provider_versions.html

[Korijn]

Is there any expected date for promotion of Ingress to stable API?

[tburow]

https://github.com/kubernetes/kubernetes/issues/74679

Pressing them as much as I can for information/schedule

[aeneasr]

I think that's the wrong approach. Upstream shouldn't force a GA release just because this provider doesn't want to implement beta features. Doing what @eveenendaal suggested is way smarter IMO.

[Korijn]

I'm not interested in pressuring/forcing anything, I intend to wait for this stuff to become stable. I was just wondering if there is any public timeline/roadmap available.

[alexsomesan]

Hi all. All the demand for this hasn't gone unnoticed. I will try to work on the Ingress resource in the next few days.

[tburow]

IMHO the fact that this has been in this state on K8S for "years" seems to be an issue. I am only pursuing getting the issue some attention as its a limiting factor for the K8S community not just this provider. Trying to find some common ground. if there is a real reason ingress was never rolled to GA then so be it, but it seems to be in the current state due to lack of attention, not technical reasoning.

[tburow]

For Ref - looks to be moving forward on the k8s side
https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/20190125-ingress-api-group.md

[lpil]

Hi @alexsomesan, was progress made in the end?

[alexsomesan]

Hi, actually as I picked this up, PR https://github.com/terraform-providers/terraform-provider-kubernetes/pull/417 showed up. So we're working on merging that.

[lpil]

Great, thank you for the update

[yzargari]

Thanks @alexsomesan, this is really excellent news! Do you think this could go into 1.6.3? Can you roughly estimate when this feature will be released?

[alexsomesan]

Given that there aren’t many changes needed in the PR and that 1.6.3 doesn’t have a planned date yet (1.6.2 just came out), I’d say chances are pretty high that this will be included in the next release.

[tdmalone]

This has now been released in 1.7.0: https://github.com/terraform-providers/terraform-provider-kubernetes/blob/master/CHANGELOG.md#170-may-22-2019
🎉 🚀

[jackivanov]

The docs were not updated

[alexsomesan]

The docs deployment was likely conflicting with the Terraform 0.12 rollout.
I will fix it.

On Thu 23. May 2019 at 16:25, Jack Ivanov [e-mail] wrote:

The docs
https://www.terraform.io/docs/providers/kubernetes/r/ingress.html were
not updated

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/terraform-providers/terraform-provider-kubernetes/issues/14?email_source=notifications&email_token=AAIL5GY3JM7UM57L4H3APV3PW2SOZA5CNFSM4DQ75OV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWCMS7A#issuecomment-495241596,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIL5G5MRJH6V4RZLDORFYDPW2SOZANCNFSM4DQ75OVQ
.

>

— Sent from my phone.