Argo: Memory Configurations for Argo Operator

Created on 25 Dec 2018 · 5Comments · Source: argoproj/argo

In docs/workflow-controller-configmap.yaml:

    # executorResources specifies the resource requirements that will be used for the executor
    # sidecar/init container. This is useful in clusters which require resources to be specified as
    # part of admission control.
    executorResources:
      requests:
        cpu: 0.1 
        memory: 64Mi
      limits:
        cpu: 0.5 
        memory: 512Mi

Do those resource settings work for almost scenes, right?
Also, above setting is for executor, not operator.
Is there any practice about resource setting for operator?
Or for ~100 jobs, how much resources need i set for operator?

question

Source

daixiang0

👍1

All 5 comments

I'm looking for documentation how to configure and/or calculate the memory (RAM) used by argo operator.

daixiang0 on 26 Dec 2018

👍1

could anyone help me?

daixiang0 on 2 Jan 2019

👍1

+1 ... would be interesting to understand what to specify for both controller as well.

Is there any more insight into those limits of 0.1 CPU + 64Mi memory for executor? Is this fixed, in which case, can it be smaller? Or should this in some way be correlated to a particular step of a workflow?

nikhilk on 4 Jan 2019

👍1

any update?

daixiang0 on 11 Feb 2019

The memory configuration required by the controller are actually dependent on how many running workflows are in your system, so there's no one size fits all. The controller uses a Kubernetes informer cache to operate on workflows and the more running workflows, then more memory is required.

As for executor resources, this also sometimes depends on the size of artifacts being downloaded. For example, it's been reported for very large artifacts (tens of GB), the memory usage is large. See:
https://github.com/argoproj/argo/issues/1322