Eksctl: `eksctl enable repo` does not work in 0.18.0 on a fresh cluster

[therealppa]

I tried to follow the tutorial to enable gitops on my cluster (freshly created with eksctl), but it does not seem to install the Helm Operator. Output:

EKSCTL_EXPERIMENTAL=true \ eksctl enable repo \ --config-file eks-cluster.yaml \ --git-url [email protected]:<redacted> \ --git-email <redacted> [ℹ] Generating manifests [ℹ] Cloning [email protected]:<redacted> Klone nach '/var/folders/x9/bnhj7dcj5dl907qgkbv70q9w0000gn/T/eksctl-install-flux-clone-738223759' ... remote: Enumerating objects: 65, done. remote: Counting objects: 100% (65/65), done. remote: Compressing objects: 100% (61/61), done. remote: Total 65 (delta 15), reused 0 (delta 0), pack-reused 0 Empfange Objekte: 100% (65/65), 22.81 KiB | 898.00 KiB/s, Fertig. Löse Unterschiede auf: 100% (15/15), Fertig. Bereits auf 'master' Ihr Branch ist auf demselben Stand wie 'origin/master'. [ℹ] Writing Flux manifests [ℹ] Applying manifests [ℹ] replaced "CustomResourceDefinition.apiextensions.k8s.io/helmreleases.helm.fluxcd.io" [ℹ] replaced "flux:Deployment.apps/helm-operator" [ℹ] replaced "flux:ServiceAccount/flux" [ℹ] replaced "ClusterRole.rbac.authorization.k8s.io/flux" [ℹ] replaced "ClusterRoleBinding.rbac.authorization.k8s.io/flux" [ℹ] replaced "flux:Deployment.apps/flux" [ℹ] replaced "flux:ServiceAccount/helm-operator" [ℹ] replaced "ClusterRole.rbac.authorization.k8s.io/helm-operator" [ℹ] replaced "ClusterRoleBinding.rbac.authorization.k8s.io/helm-operator" [ℹ] Waiting for Helm Operator to start [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [!] Helm Operator is not ready yet (Could not create a dialer: Could not get pod name: Could not find pod for selector: labels "name in (flux-helm-operator)"), retrying ... [✖] You may find the local clone of [email protected]:<redacted> used by eksctl at /var/folders/x9/bnhj7dcj5dl907qgkbv70q9w0000gn/T/eksctl-install-flux-clone-738223759 [ℹ] Error: timed out waiting for Helm Operator's pod to be created

cluster config:
````

An example of ClusterConfig object compatible with the "app-dev" quickstart

# profile. See also: https://github.com/weaveworks/eks-quickstart-app-dev

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name:
region: eu-central-1

managedNodeGroups:

• name: ng-1
  instanceType: t3a.large
  minSize: 1
  maxSize: 2
  iam:
  withAddonPolicies:
  albIngress: true
  autoScaler: true
  cloudWatch: true
  ````

This might also be interestion (after the failed command):

kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE flux flux-85b668c868-bfk4p 1/1 Running 0 11m flux helm-operator-74546bd6f5-n4n82 1/1 Running 0 11m flux memcached-689f7846dc-smqqt 1/1 Running 0 11m kube-system aws-node-vn2fd 1/1 Running 0 17m kube-system coredns-5b6dbb4b59-69jtb 1/1 Running 0 22m kube-system coredns-5b6dbb4b59-lmnfk 1/1 Running 0 22m kube-system kube-proxy-6lmbf 1/1 Running 0 17m

This is with eksctl 0.18.0. It does work with eksctl 0.17.0. The run I pasted here uses a repo that was created by 0.17.0 but it does not work with an empty repo either.

[sayboras]

Related to https://github.com/weaveworks/eksctl/pull/2117

[therealppa]

@sayboras Yes, that looks like the same problem :) Only looked through the issues, not the PRs.

[sayboras]

@therealppa no worry, I just want to add the link between this issue and PR, so that it's easier to follow as well. Hope that #2117 will land in soon :100:

[therealppa]

Any ideas for a workaround? I tried creating a pod with the name flux-helm-operator but it seems I'm missing something.

[marclamberti]

I'm very sad, same error,
First use of GitOps with eksctl :'(

[sayboras]

@therealppa @marclamberti

Sorry to hear that. I just checked the release notes, you can try eksctl version 0.17.0 or before, obviously, it didn't have support for some additional feature.

Personally, I usually checked out the branch and build eksctl binary directly, however, this one might require some work if you don't have a development environment ready.

[greg-cook]

@therealppa @marclamberti
...
Personally, I usually checked out the branch and build eksctl binary directly ...

This is what I did to workaround the issue 😄

[martina-if]

Hi @therealppa Thanks for reporting this. As it was mentioned before there is a fix for this in the pipeline. Can you please confirm that the helm-operator components were installed in your cluster in the flux namespace?

[therealppa]

Can you please confirm that the helm-operator components were installed in your cluster in the flux namespace?

This was installed:

kubectl get pod --all-namespaces                                                                                                                                                                               NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE
flux          flux-85b668c868-bfk4p            1/1     Running   0          11m
flux          helm-operator-74546bd6f5-n4n82   1/1     Running   0          11m
flux          memcached-689f7846dc-smqqt       1/1     Running   0          11m
kube-system   aws-node-vn2fd                   1/1     Running   0          17m
kube-system   coredns-5b6dbb4b59-69jtb         1/1     Running   0          22m
kube-system   coredns-5b6dbb4b59-lmnfk         1/1     Running   0          22m
kube-system   kube-proxy-6lmbf                 1/1     Running   0          17m

[martina-if]

Fixed by https://github.com/weaveworks/eksctl/pull/2117
We will release this as a hotfix in 0.19.0-rc.1