Argo-cd: ARM Docker Builds

[phillebaba]

I have a Kubernetes cluster with some ARM based nodes. I wanted to run Argo CD on them. The problem is that there are no Docker images for ARM devices. So I wen along and created a fork to verify that it would be possible to build ArgoCD for ARM, which it turns out it is.

The changes are not optimal as it breaks any existing build for x86 as it was more of a POC. Additionally i had some issue building aws-iam-authenticator, which i skipped for now.
https://github.com/phillebaba/argo-cd/tree/feature/arm

I pushed the resulting image here.
https://cloud.docker.com/u/phillebaba/repository/docker/phillebaba/argo-cd

Is there any interest to take these changes so that there would be official support for ARM? I have not spent any time looking at the existing CI but my guess is that it would require some changes.

[alexec]

Good one for community contribution!

[phillebaba]

I have updated the docker image for the latest release of argocd.
https://cloud.docker.com/repository/docker/phillebaba/argocd

How can we proceed so that I do not need to sync with every release. I have been meaning to join the community meeting but something keeps getting in the way.

[alexec]

You’d probably need to create a new make target that produces the image you need.

[mcanevet]

@phillebaba could you please build an image for 1.3.0rc?

[phillebaba]

@mcanevet There is a build for 1.3.0rc available now. I rebased the branch "feature/arm" on the tag so the ARM build should be identical to the amd64 build.

@alexec Sadly it is not that simple. The major issue right now is that some of the dependencies need to be compiled for ARM, the good news is that its Go so its easy yet time consuming to do it.

Looking at the Makefile this would not be an issue if they were only built in the docker image but it seems like the image rule either builds things on the local machine or in the docker image build step. This creates some complexity in the process.

I have some extra time during the weekend so I will create a suggestion of how we could solve this, but it may change some things in the current build solution which others may not like.

[phillebaba]

I added a Makefile.arm file to keep things separate. What I changed there is that I added a push rule that uses buildx to build and push the resulting image.
https://github.com/phillebaba/argo-cd/tree/feature/arm

This should solve most of the problems if we can accept the building from source in the Dockerfile. The image rule could be solved by setting the "building nodes" architecture to GOARCH so that the build would work. The idea is that you would build locally when developing but cross compile when pushing images to be used on other platforms that are not your own local dev machine. So if we can accept this I will go along and make the necessary changes to the other rules to make it work.

Note that this is just a POC and is a reflection of my idea of how development should be done. Image builds should be done locally and run locally. Everything else should be pushed for multiple architectures so it can be tested on multiple platforms.

[keithchester]

I was looking to see if argo CD worked on ARM (useful for our IoT devices) and came across this issue. Was there any movement on this?

[phillebaba]

Nope not really, I have been running my fork in my Raspberry Pi cluster for the last couple of months without any issues. I update the fork when I want a new feature and when I have time to do so.

Other than that I am waiting for a response with information about how we would move forward.

[KonKube]

I am also looking for an integration for arm builds.
I just tried it myself and have added as much arm binaries of tools as possible.

Tried to build them for arm, if any tool does not have any arm binaries. (With no luck :D)

Nice feature of docker build-x (https://docs.docker.com/buildx/working-with-buildx/) is the possibility to build for multiple platforms.
Also the use of platform variables, which I included to the argo-cd/hack/installers parts and the docker file.

ARG TARGETPLATFORM
ARG BUILDPLATFORM

https://github.com/KonKube/argo-cd/tree/v1.4.2-arm64

Would appreciate, if we could create a dockerfile etc. which keeps all nice and simple.

[pisymbol]

@phillebaba How do you install? Import your image into a private repo and then edit install.yaml accordingly?

I really want this as well (I too have a Pi cluster to maintain!).

[phillebaba]

I am a heavy Kustomize user, so I have created a manifests repository that contains kustomization files for different projects. Here is the one for ArgoCD.

I then use that base kustomization file and patch in my own ARM image instead of the default ArgoCD image. The kustomization file I am using to bootstrap my RPi cluster can be found here.

[danieldabate]

I could also make use of the ARM images if this gets implemented. For now I'm using phillebaba's 1.3.6 image as a workaround.

[pisymbol]

What is the install instructions for getting Argo CD on arm?

[danieldabate]

I don't know if this is what you are asking for, but you can find the Docker images phillebaba built for ARM here: https://hub.docker.com/r/phillebaba/argocd/tags

[pisymbol]

@danieldabate Put simply, how do you install those images?

[danieldabate]

Just follow the Getting Started doc but change the argoproj/argocd:v1.4.2 image from install.yaml to phillebaba/argocd:v1.3.6.

https://argoproj.github.io/argo-cd/getting_started/

[pisymbol]

What about the command line tool? @danieldabate

[danieldabate]

I don't know if the cli is built for arm, I run it on other computer, not directly on the RPI.

[phillebaba]

The cli inside of the docker image should be compiled for ARM. If you want it on another computer you would either need to compile it yourself or copy it from the docker image.

[pisymbol]

Thanks @phillebaba ! Looks like everything works.

[phillebaba]

I have pushed a new image for v1.4.2 but I wont have time to verify it during the coming week. So if anyone tries it out and finds any issues feel free to tell me.

I guess I will have to continue doing this for as long as we can't figure out how to resolve this issue.

[danieldabate]

I can confirm the new image is working great. Thanks!

[sprinkaan88]

@danieldabate && @phillebaba
Thanks for the effort gents.
I am trying to get this installed on my RPI.

I took the installer from https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Replaced the image as suggested : argoproj/argocd:latest to phillebaba/argocd:v1.4.2
Apply said file.

argocd-dex-server | argocd-redis | argocd-repo-server => starts up with Running status

argocd-server (Not starting) logs :

time="2020-02-27T10:33:29Z" level=info msg="Starting configmap/secret informers"
I0227 10:34:00.000537       1 trace.go:116] Trace[1298498081]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/util/settings/settings.go:560 (started: 2020-02-27 10:33:29.998807184 +0000 UTC m=+0.166843968) (total time: 30.001550223s):
Trace[1298498081]: [30.001550223s] [30.001550223s] END
E0227 10:34:00.000666       1 reflector.go:123] github.com/argoproj/argo-cd/util/settings/settings.go:560: Failed to list *v1.Secret: Get https://10.96.0.1:443/api/v1/namespaces/argocd/secrets?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0227 10:34:00.001478       1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/util/settings/settings.go:556 (started: 2020-02-27 10:33:30.000016249 +0000 UTC m=+0.168052959) (total time: 30.001291484s):
Trace[2019727887]: [30.001291484s] [30.001291484s] END
E0227 10:34:00.001560       1 reflector.go:123] github.com/argoproj/argo-cd/util/settings/settings.go:556: Failed to list *v1.ConfigMap: Get https://10.96.0.1:443/api/v1/namespaces/argocd/configmaps?labelSelector=app.kubernetes.io%2Fpart-of%3Dargocd&limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0227 10:34:31.002207       1 trace.go:116] Trace[1427131847]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/util/settings/settings.go:560 (started: 2020-02-27 10:34:01.000985545 +0000 UTC m=+31.169022884) (total time: 30.001123786s):
Trace[1427131847]: [30.001123786s] [30.001123786s] END
E0227 10:34:31.002252       1 reflector.go:123] github.com/argoproj/argo-cd/util/settings/settings.go:560: Failed to list *v1.Secret: Get https://10.96.0.1:443/api/v1/namespaces/argocd/secrets?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0227 10:34:31.003115       1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/util/settings/settings.go:556 (started: 2020-02-27 10:34:01.002075407 +0000 UTC m=+31.170112673) (total time: 30.000978009s):
Trace[939984059]: [30.000978009s] [30.000978009s] END
E0227 10:34:31.003149       1 reflector.go:123] github.com/argoproj/argo-cd/util/settings/settings.go:556: Failed to list *v1.ConfigMap: Get https://10.96.0.1:443/api/v1/namespaces/argocd/configmaps?labelSelector=app.kubernetes.io%2Fpart-of%3Dargocd&limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout

argocd-application-controller (Not starting) logs :

time="2020-02-27T10:36:33Z" level=info msg="appResyncPeriod=3m0s"
time="2020-02-27T10:36:33Z" level=info msg="Application Controller (version: v1.4.2+762a500.dirty) starting (namespace: argocd)"
I0227 10:37:03.938148       1 trace.go:116] Trace[1298498081]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/controller/appcontroller.go:410 (started: 2020-02-27 10:36:33.93656128 +0000 UTC m=+0.153466060) (total time: 30.00139821s):
Trace[1298498081]: [30.00139821s] [30.00139821s] END
E0227 10:37:03.938273       1 reflector.go:123] github.com/argoproj/argo-cd/controller/appcontroller.go:410: Failed to list *v1alpha1.AppProject: Get https://10.96.0.1:443/apis/argoproj.io/v1alpha1/namespaces/argocd/appprojects?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0227 10:37:03.938393       1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:github.com/argoproj/argo-cd/controller/appcontroller.go:409 (started: 2020-02-27 10:36:33.936546484 +0000 UTC m=+0.153451337) (total time: 30.001587394s):
Trace[2019727887]: [30.001587394s] [30.001587394s] END
E0227 10:37:03.938468       1 reflector.go:123] github.com/argoproj/argo-cd/controller/appcontroller.go:409: Failed to list *v1alpha1.Application: Get https://10.96.0.1:443/apis/argoproj.io/v1alpha1/namespaces/argocd/applications?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout

Running one master and one worker node, both with k8s v1.17.3

Any idea why I would be getting timeouts getting to the api server ?