Argo: `argo submit --priority` doesn't effect workflow's priority

[kounoike]

Is this a BUG REPORT or FEATURE REQUEST?: BUG REPORT

What happened: Running workflow has priority=0, so workflows runs randomly sequence.

What you expected to happen: priority is provided by command line. and workflows runs ordered by priority.

How to reproduce it (as minimally and precisely as possible):

$ argo submit --output yaml --priority 12345 https://raw.githubusercontent.com/argoproj/argo/master/examples/hello-world.yaml && argo list --output wide

I got

metadata:
  creationTimestamp: "2019-06-14T07:41:07Z"
  generateName: hello-world-
  generation: 1
  name: hello-world-jsqdh
  namespace: default
  resourceVersion: "4035578"
  selfLink: /apis/argoproj.io/v1alpha1/namespaces/default/workflows/hello-world-jsqdh
  uid: c58b2549-8e77-11e9-a9c8-4ccc6a46d4ce
spec:
  arguments: {}
  entrypoint: whalesay
  priority: 12345
  templates:
  - container:
      args:
      - hello world
      command:
      - cowsay
      image: docker/whalesay:latest
      name: ""
      resources: {}
    inputs: {}
    metadata: {}
    name: whalesay
    outputs: {}
status:
  finishedAt: null
  startedAt: null
NAME                STATUS    AGE   DURATION   PRIORITY   P/R/C   PARAMETERS
hello-world-jsqdh   Running   1s    1s         0          1/0/0   

It seems as: argo submit output shows priority=12345, but argo list shows priority=0

Environment:

• Argo version:

$ argo version
argo: v2.3.0
  BuildDate: 2019-05-20T22:11:23Z
  GitCommit: 88fcc70dcf6e60697e6716edc7464a403c49b27e
  GitTreeState: clean
  GitTag: v2.3.0
  GoVersion: go1.11.5
  Compiler: gc
  Platform: darwin/amd64

• Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: 2018-11-26T14:38:32Z
  compiler: gc
  gitCommit: 637c7e288581ee40ab4ca210618a89a555b6e7e9
  gitTreeState: clean
  gitVersion: v1.10.11
  goVersion: go1.9.3
  major: "1"
  minor: "10"
  platform: darwin/amd64
serverVersion:
  buildDate: 2019-03-25T15:19:22Z
  compiler: gc
  gitCommit: 2166946f41b36dea2c4626f90a77706f426cdea2
  gitTreeState: clean
  gitVersion: v1.13.5
  goVersion: go1.11.5
  major: "1"
  minor: "13"
  platform: linux/amd64

Other debugging information (if applicable):

• workflow result:

$ argo get hello-world-jsqdh
Name:                hello-world-jsqdh
Namespace:           default
ServiceAccount:      default
Status:              Succeeded
Created:             Fri Jun 14 16:41:07 +0900 (6 minutes ago)
Started:             Fri Jun 14 16:41:07 +0900 (6 minutes ago)
Finished:            Fri Jun 14 16:41:17 +0900 (6 minutes ago)
Duration:            10 seconds

STEP                  PODNAME            DURATION  MESSAGE
 ✔ hello-world-jsqdh  hello-world-jsqdh  9s      

kubectl get pods -o yaml shows:

apiVersion: v1
items:
- apiVersion: v1
  kind: Pod
  metadata:
    annotations:
      cni.projectcalico.org/podIP: 10.42.0.24/32
      workflows.argoproj.io/node-name: hello-world-jsqdh
      workflows.argoproj.io/template: '{"name":"whalesay","inputs":{},"outputs":{},"metadata":{},"container":{"name":"","image":"docker/whalesay:latest","command":["cowsay"],"args":["hello
        world"],"resources":{}},"archiveLocation":{"s3":{"endpoint":"s3.amazonaws.com","bucket":"kounoike-argo-test","accessKeySecret":{"name":"argo-s3-credentials","key":"accessKey"},"secretKeySecret":{"name":"argo-s3-credentials","key":"secretKey"},"key":"default-artifacts/hello-world-jsqdh/hello-world-jsqdh"}}}'
    creationTimestamp: 2019-06-14T07:41:07Z
    labels:
      workflows.argoproj.io/completed: "true"
      workflows.argoproj.io/workflow: hello-world-jsqdh
    name: hello-world-jsqdh
    namespace: default
    ownerReferences:
    - apiVersion: argoproj.io/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: Workflow
      name: hello-world-jsqdh
      uid: c58b2549-8e77-11e9-a9c8-4ccc6a46d4ce
    resourceVersion: "4035613"
    selfLink: /api/v1/namespaces/default/pods/hello-world-jsqdh
    uid: c58d432c-8e77-11e9-a9c8-4ccc6a46d4ce
  spec:
    containers:
    - args:
      - hello world
      command:
      - cowsay
      image: docker/whalesay:latest
      imagePullPolicy: Always
      name: main
      resources: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-6vlv7
        readOnly: true
    - args:
      - wait
      command:
      - argoexec
      env:
      - name: ARGO_POD_NAME
        valueFrom:
          fieldRef:
            apiVersion: v1
            fieldPath: metadata.name
      image: argoproj/argoexec:v2.2.1
      imagePullPolicy: IfNotPresent
      name: wait
      resources: {}
      securityContext:
        privileged: false
        procMount: Default
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /argo/podmetadata
        name: podmetadata
      - mountPath: /var/lib/docker
        name: docker-lib
        readOnly: true
      - mountPath: /var/run/docker.sock
        name: docker-sock
        readOnly: true
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-6vlv7
        readOnly: true
    dnsPolicy: ClusterFirst
    enableServiceLinks: true
    nodeName: ubuntu-gpu-1
    restartPolicy: Never
    schedulerName: default-scheduler
    securityContext: {}
    serviceAccount: default
    serviceAccountName: default
    terminationGracePeriodSeconds: 30
    tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
    volumes:
    - downwardAPI:
        defaultMode: 420
        items:
        - fieldRef:
            apiVersion: v1
            fieldPath: metadata.annotations
          path: annotations
      name: podmetadata
    - hostPath:
        path: /var/lib/docker
        type: Directory
      name: docker-lib
    - hostPath:
        path: /var/run/docker.sock
        type: Socket
      name: docker-sock
    - name: default-token-6vlv7
      secret:
        defaultMode: 420
        secretName: default-token-6vlv7
  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: 2019-06-14T07:41:08Z
      reason: PodCompleted
      status: "True"
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: 2019-06-14T07:41:08Z
      reason: PodCompleted
      status: "False"
      type: Ready
    - lastProbeTime: null
      lastTransitionTime: 2019-06-14T07:41:08Z
      reason: PodCompleted
      status: "False"
      type: ContainersReady
    - lastProbeTime: null
      lastTransitionTime: 2019-06-14T07:41:07Z
      status: "True"
      type: PodScheduled
    containerStatuses:
    - containerID: docker://3f837e22699a682c52fcf4ddba91d463c72b7f7dedac3e1cc758e076f7216352
      image: docker/whalesay:latest
      imageID: docker-pullable://docker/whalesay@sha256:178598e51a26abbc958b8a2e48825c90bc22e641de3d31e18aaf55f3258ba93b
      lastState: {}
      name: main
      ready: false
      restartCount: 0
      state:
        terminated:
          containerID: docker://3f837e22699a682c52fcf4ddba91d463c72b7f7dedac3e1cc758e076f7216352
          exitCode: 0
          finishedAt: 2019-06-14T07:41:13Z
          reason: Completed
          startedAt: 2019-06-14T07:41:13Z
    - containerID: docker://8ba794c181f4ed8f249781ad000a4407fa7f8414719f08bcb08f3d83c42de6c9
      image: argoproj/argoexec:v2.2.1
      imageID: docker-pullable://argoproj/argoexec@sha256:9b12553aa7dccddc88c766d3dd59f4e8758acbd82ceef9e7aedc75f09934480a
      lastState: {}
      name: wait
      ready: false
      restartCount: 0
      state:
        terminated:
          containerID: docker://8ba794c181f4ed8f249781ad000a4407fa7f8414719f08bcb08f3d83c42de6c9
          exitCode: 0
          finishedAt: 2019-06-14T07:41:16Z
          reason: Completed
          startedAt: 2019-06-14T07:41:14Z
    hostIP: 10.10.32.20
    phase: Succeeded
    podIP: 10.42.0.24
    qosClass: BestEffort
    startTime: 2019-06-14T07:41:08Z
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

There is no priority field.

[jdeprin]

I wanted to update this in case it is helpful... when defining the priority directly in the workflow manifest and not as a flag on the cli, a priority is not applied.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
  namespace: argo-events
spec:
  entrypoint: whalesay
  priority: 10
  templates:
  - container:
      args:
      - hello world
      command:
      - cowsay
      image: docker/whalesay:latest
    name: whalesay

argo -n argo submit workflow.yaml
kubectl -n argo describe workflow hello-world-q9j97

Name:         hello-world-q9j97
Namespace:    argo
Labels:       workflows.argoproj.io/completed=true
              workflows.argoproj.io/phase=Succeeded
Annotations:  <none>
API Version:  argoproj.io/v1alpha1
Kind:         Workflow
Metadata:
  Creation Timestamp:  2019-10-02T17:12:15Z
  Generate Name:       hello-world-
  Generation:          4
  Resource Version:    168932231
  Self Link:           /apis/argoproj.io/v1alpha1/namespaces/argo-events/workflows/hello-world-q9j97
  UID:                 c84bf063-e537-11e9-bb1d-0a871bab8274
Spec:
  Arguments:
  Entrypoint:  whalesay
  Templates:
    Arguments:
    Container:
      Args:
        hello world
      Command:
        cowsay
      Image:  docker/whalesay:latest
      Name:
      Resources:
    Inputs:
    Metadata:
    Name:  whalesay
    Outputs:
Status:
  Finished At:  2019-10-02T17:12:23Z
  Nodes:
    Hello - World - Q 9 J 97:
      Display Name:   hello-world-q9j97
      Finished At:    2019-10-02T17:12:23Z
      Id:             hello-world-q9j97
      Name:           hello-world-q9j97
      Phase:          Succeeded
      Started At:     2019-10-02T17:12:15Z
      Template Name:  whalesay
      Type:           Pod
  Phase:              Succeeded
  Started At:         2019-10-02T17:12:15Z
Events:               <none>

No priority field is listed.

[melihme]

Likewise, I'm adding parallelism to Workflow manifest and it broke the deployment.

Can anybody refer a document/example to clarify these options?

[fabio-rigato]

Hi guys!
any updates about that?
Thank you for your effort.

[alexec]

Can you see if this is still an issue in v2.5? Is so - please re-open.