Argo: Allow setting workflow parameter values using ConfigMap

[erhhung]

Summary

There is currently no way to set a workflow parameter value (for both global spec.arguments.parameters and steps[].arguments.parameters) using the valueFrom.configMapKeyRef construct in the same way env variables can be defined.

It would be nice to be able to use a value from a ConfigMap as the default and still allow it to be overridden using the CLI -p option.

Motivation

I want to define a custom Docker registry endpoint in a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: environment-config
  namespace: worker
data:
  docker.registry: '123456789012.dkr.ecr.us-west-1.amazonaws.com'

And be able to use it in a workflow:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: myworkflow-
spec:
  entrypoint: step1
  arguments:
    parameters:
    - name: registry
      valueFrom:
        configMapKeyRef:
          name: environment-config
          key: docker.registry
  templates:
  - name: step1
    container:
      image: '{{workflow.parameters.registry}}/step1:latest'
      command: [./start.sh]

Proposal

Same behavior as valueFrom.configMapKeyRef for environment variables.

In this specific example of customizing the Docker registry endpoint, if there's another better way without duplicating the actual value in multiple places, that would also be appreciated.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[simster7]

Great suggestion. I'll look into this

[simster7]

Looked into this. Problem with the suggestion is that valueFrom is used only to retrieve values for _output_ parameters. Arguments may only use parameters.value to specify the value.

Good news is that you can use a resource template to get the value from your ConfigMap today:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: get-cm-value-
spec:
  entrypoint: entry

  templates:
    - name: entry
      steps:
        - - name: get-cm-value
            template: get-cm-value
        - - name: use-cm-value
            template: use-cm-value
            arguments:
              parameters:
                - name: docker-val
                  value: "{{steps.get-cm-value.outputs.parameters.docker-val}}"

    - name: get-cm-value
      resource:
        action: get
        # This specifies which ConfigMap to look at
        manifest: |
          apiVersion: v1
          kind: ConfigMap
          metadata:
            name: environment-config
      outputs:
        parameters:
          - name: docker-val
            valueFrom:
              jsonPath: '{.data.docker-registry}'    # Pick which value you want here


    - name: use-cm-value
      inputs:
        parameters:
          - name: docker-val
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo {{inputs.parameters.docker-val}}"]

[simster7]

Closing as solved? Feel free to comment if not

[erhhung]

Sorry I wasn't monitoring the responses closely.

@simster7, I don't quite understand your explanation about valueFrom being only used to retrieve _output_ parameters, as if that's logically the only way it's supposed to work.

If I can inject _input_ environment variables (container.env) using valueFrom.configMapKeyRef or valueFrom.secretKeyRef (see secrets.yaml), or even simpler envFrom.configMapRef, then I don't see why logically I can't also fetch ConfigMap values for inputs.parameters or workflow.parameters.

Your example using a resource template is at best a workaround because it's extremely verbose (see all the places docker-val is used). It requires an extra boilerplate template step on every workflow (because I'd need the registry URL in every workflow) that will also appear in the dashboard graph. Sure, it's a low-level, generic approach to perform any action on any K8s resource, but then why there exists a convenience "function" for injecting environment variables? It's this imbalance of expectations that I see as the main problem.

[andizzle]

Bump. Can we re-open this as a feature request?

[alexec]

@andizzle re-opened. Would you like to submit a PR?

[firecast]

@alexec I've created a PR for this feature