External-dns: Multi-cluster weighted records for canary cluster deployments

[carlosjgp]

What would you like to be added:
2 ExternalDNS deployments on 2 Kubernetes clusters to manage the weighted DNS entries for the same hostname

This feature is mentioned in this issue comment.

Canary release on a new Kubernetes cluster when adding or changing a critical component either related to the Kubernetes architecture like the CNI plugin, CSI plugin, new K8s version...
or the infrastructure of that runs on the cluster like service mesh, API gateway, distroless containers...

Why is this needed:
this will allow migrating traffic from one cluster into another by changing the Ingres manifest

Currently for the regular setup to perform canary cluster deployment where ingress traffic is weighted little by little into the new cluster requires a proxy in front of the clusters with a wildcard DNS or managing all the DNS entries with weighted DNS entries when this is possible (like AWS Route53)

[paalkr]

@spohner You implemented this feature, didn't you?

[spohner]

We use the txt-owner-id and txt-prefix with weighted entries set on the ingress as routing-policies (https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/aws.md#routing-policies) with the AWS provider. Which seems to work fine for our use-case.

[yokawasa]

@spohner
I am using ExternalDNS with alb-ingress-controller, and I want to achieve the same scenario.

CORRECTION:
I tried with the following ingress manifest expecting to configure weighted policy for my dns record, and I confirmed routing-policies works.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthcheck
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
    external-dns.alpha.kubernetes.io/hostname: test.mydomain.com
    external-dns.alpha.kubernetes.io/set-identifier: cluster01
    external-dns.alpha.kubernetes.io/aws-weight: "5"

This below was an original question I made but please ignore this. I didn't enclose the value for external-dns.alpha.kubernetes.io/aws-weight in double quotes. Sorry for the mess.
Question: Would the routing-policies work for my case like using ExternalDNS with alb-ingress-controller?
I tried with the following ingress manifest expecting to configure weighted policy for my dns record.
However, I came up with the error like this when I apply the manifest. Without external-dns.alpha.kubernetes.io/aws-weight annotation, I don't get an error.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthcheck
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
    external-dns.alpha.kubernetes.io/hostname: test.mydomain.com
    external-dns.alpha.kubernetes.io/set-identifier: cluster01
    external-dns.alpha.kubernetes.io/aws-weight: 5    <<<--- It was a mistake

error: unable to decode "api-gateway": resource.metadataOnlyObject.ObjectMeta: v1.ObjectMeta.Annotations: ReadString: expects " or n, but found 5, error found in #10 byte of ...|-weight":50,"externa|..., bigger context ...|2","external-dns.alpha.kubernetes.io/aws-weight":50,"external-dns.alpha.kubernetes.io/hostname":"api|...

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[seanmalloy]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[seanmalloy]

/remove-lifecycle stale