Flux: Add the oc binary from openshift as different ctl

[leroy0211]

Describe the feature
Openshift has it's own resource definitions like Route. Right now when those openshift resources are detected, kubectl does not apply those.
It would be a great addition to add the oc binary, and let the user choose which binary is being used.

What would the new user story look like?

1. User starts up Flux using the --kubernetes-bin=openshift
2. Flux now uses the oc binary instead of the kubectl binary when applying new configuration

Expected behavior
When custom resource definitions of Openshift are applied like Route. Those become available in the cluster.

Right now, those resources are not even applied, and no error is being logged. However, when applying those manual with kubectl apply -f [file] some errors occur, but if you apply them with oc apply -f [file], everything is fine.

[squaremo]

There is already a flag for pointing at an alternative binary -- --kubernetes-kubectl. If oc accepts the same flags as kubectl, you could try mounting (er, copying) an oc binary into the fluxd container, and supplying the path to it using --kubernetes-kubectl.

[leroy0211]

Oh thanks! Didn't know that. I'm just starting with FluxCD.
Do you think we need to create a new image with the oc binary in it, or is there another solution too?

If the only way to do this is by creating our own image and change the values.yaml to point to our image that would be good too. But I would prefer to use the official images provided by the helm chart.

[leroy0211]

I was trying to do this with the helm chart, but the --kubernetes-kubectl flag is not yet available in the helm chart.

[update]
Ah there is an additionalArgs value, which can be used for this.

[goober]

@leroy0211 Since the image is based on Alpine it will not work out of the box by only downloading the oc client during image creation time. The oc client is not statically compiled which makes it dependent on glibc, which in turn is not available in the alpine image.

The workaround that we use is to create a wrapper script for the client with the following content:

#!/bin/bash

# Fix for running the openshift cli on alpine based images.
# See https://github.com/openshift/origin/issues/18942 for details
/lib/ld-musl-x86_64.so.1 --library-path /lib /usr/local/bin/oc-original $@

Or do you have another solution that worked for you?

[leroy0211]

@goober I've installed glibc with the following command and download the oc binary.

 wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub \
    && wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.28-r0/glibc-2.28-r0.apk \
    && /sbin/apk add glibc-2.28-r0.apk

However, I don't think everything works because an error occured that the api resource Route was not found. I had to specify the exact apiVersion route.openshift.io/v1 instead of v1 as the openshift documentation says. After that change, it worked.