FEATURE REQUEST:
I propose to have named parallelism, and by that I mean the ability to define several parallelism limits, and let different templates define which parallelism limit it consumes.
Why
We have several different tasks scattered around in the workflow which consume some limit resource, e.g. a database with limited capacity. We want maximum parallelism in general, but have a global limit of e.g. 10 to the database so it does crash. The different tasks does not fit after each other in a steps list.
Concrete proposals
I have 3 different proposals, in increased complexity
Proposal 1
Each task consumes a single parallelism-limit:
```...
spec:
parallelism-limits:
*Proposal 2*
Each task can consume 1 of several limits:
```...
spec:
parallelism-limits:
- name: influx-limit
value: 10
- name: sql-limit
value: 4
templates:
- name: some-thingy-which-needs-both-influx-and-sql
parallelism:
- influx-limit
sql-limit # So at most 4 of this task can run at the same time
...
Proposal 3
Each task can consume x of several limits:
```...
spec:
parallelism-limits:
I'm not sure if this is relevant, but I'd be quite interested in a global limit that works across workflows if that isn't already possible. That is, submitting two workflows, and having them essentially _share a semaphore_ (for lack of better phrasing).
This sounds like #2550 . Closing and duplicate.
Most helpful comment
I'm not sure if this is relevant, but I'd be quite interested in a global limit that works across workflows if that isn't already possible. That is, submitting two workflows, and having them essentially _share a semaphore_ (for lack of better phrasing).