Argo: imagePullSecrets not working as expected

[David-Development]

Feature Request: Docs

What happened:

I'm trying to deploy argo onto a kubernetes cluster (RKE) with more than one node where my docker images are stored in a private registry. Argo already supports the "imagePullSecrets" flag. However I can't find any documentation/examples on how to do it. I think it would be great to have a small example that shows how to use private registries in Kubernetes / with Argo. Would it be possible to set the "imagePullSecrets" once for the whole workflow? It's cumbersome to define it for each step of the workflow.

My workflow config (works)

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: training-
spec:
  entrypoint: fct
  ...
  templates:
  - name: fct
    imagePullSecrets: docker-registry-secret
    container:
      image: registry.gitlab.xxx:4567/xxx:0.1.3

However, when I try to define it "globally", argo gives me the error: "...cannot unmarshal string into Go struct field WorkflowSpec.imagePullSecrets of type []v1.LocalObjectReference..." even though I got this idea from https://github.com/argoproj/argo/issues/940.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: training-
spec:
  entrypoint: pre-processing
  imagePullSecrets: docker-registry-secret
  ...

This is the command I used to create the docker secret in the registry

# Create secret in Kubernetes
kubectl create secret docker-registry docker-registry-secret --docker-server=registry.gitlab.xxx:4567 --docker-username=xxx --docker-password="xxx" --docker-email=xxx

Environment:

• Argo version:

$ argo version 

argo: v2.2.0
BuildDate: 2018-08-30T08:52:05Z
...

• Kubernetes version :

$ kubectl version -o yaml

clientVersion:
  buildDate: 2018-09-09T18:02:47Z
  compiler: gc
  gitCommit: a4529464e4629c21224b3d52edfe0ea91b072862
  gitTreeState: clean
  gitVersion: v1.11.3
  goVersion: go1.10.3
  major: "1"
  minor: "11"
  platform: linux/amd64
serverVersion:
  buildDate: 2018-08-07T23:08:19Z
  compiler: gc
  gitCommit: bb9ffb1654d4a729bb4cec18ff088eacc153c239
  gitTreeState: clean
  gitVersion: v1.11.2
  goVersion: go1.10.3
  major: "1"
  minor: "11"
  platform: linux/amd64

[jessesuen]

@David-Development this manifest is wrong:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: training-
spec:
  entrypoint: pre-processing
  imagePullSecrets: docker-registry-secret # << should be a list

imagePullSecrets should be a list. So this is the correct way:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: training-
spec:
  entrypoint: pre-processing
  imagePullSecrets:
  - docker-registry-secret

I actually expect there to be a linting problem when you attempted to submit the bad one, since it's a mismatched type.

[David-Development]

@jessesuen Thank you for your example. Yes, my example was giving me some lint errors. But submitting the job worked fine though.

If I run lint with the changes you proposed, it's giving me the following error:

FATA[0000] speech-training.yaml failed to parse: error unmarshaling JSON: while decoding
JSON: json: cannot unmarshal string into Go struct field WorkflowSpec.imagePullSecrets
of type v1.LocalObjectReference

Any idea how to fix this?

[jessesuen]

Sorry I gave an incorrect solution. It should have a name field. This should work:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: training-
spec:
  entrypoint: pre-processing
  imagePullSecrets:
  - name: docker-registry-secret

[David-Development]

@jessesuen Awesome, thank you! It's working now. Argo is a great workflow engine!

One suggestions: Maybe your example could be linked somewhere in the docs under "custom/private registry"?

[jessesuen]

Sure, I just added an example of a workflow using imagePullSecrets.