Argo-cd: Support generateName for application resources

[jessesuen]

A common request is to support generateName in resources. Although kubectl apply does not work with generateName, Argo CD could have behavior that when it sees a resource with generateName instead of name, it could decide to perform create instead.

Note that resources created in this manner, would immediately cause the application to be OutOfSync, since Argo CD would consider these as "extra" resources that need to be pruned. To mitigate this, the user could use this feature to prevent the extra resource from contributing to the overall OutOfSync condition of the application as a whole.

With this feature, Argo CD could be used to trigger job runs by simply performing a sync.

Some areas of concern:

1. how would auto-sync behave with this feature
2. if the resource is a Job or Workflow, the sync operation should probably not wait until those resources complete (unlike resource hooks).
3. how would this work with hook-weights ?
4. diffing will not work on generateName objects.

Also to note: it is already possible to have Argo CD create resources using generateName, but those resources need to use the argocd.argoproj.io/hook annotation. e.g.:

metadata:
  generateName: my-job-
  annotations:
    argocd.argoproj.io/hook: Sync

However, using resource hooks has the following limitations:

1. Live hook objects are not considered part of the application, and thus not candidates for pruning. They will still be presented in the UI.
2. With resource hooks, Job/Workflow/Pod objects will block a Sync operation from completing until the Job/Workflow/Pod completes. So very long lived jobs/pods/workflows would prevent new argocd app sync from occurring. This would be undesirable for someone who just wants to kick off the job asynchronously

[kwladyka]

What are reasons kubectl apply not work with generateName? I mean while it is not supported natively maybe there is good reason for it.

My use case for it:
I want to create argocd Application which will have only 1 job. This job will be GitOps pipelines for Concourse (cicd tool). So after each change in pipelines and push to git it will run and be sure to update all pipelines configurations in Concourse. In this way I can achieve GitOps for pipelines.

Alternatively I can run this in Concourse to update Concourse pipelines ;) The border of where it should be done is abstractive ;)

[jessesuen]

What are reasons kubectl apply not work with generateName? I mean while it is not supported natively maybe there is good reason for it.

You can read the discussion here: https://github.com/kubernetes/kubernetes/issues/44501

The resolution was to document this limitation, rather than have kubectl apply handle generateName

[jessesuen]

I want to create argocd Application which will have only 1 job. This job will be GitOps pipelines for Concourse (cicd tool). So after each change in pipelines and push to git it will run and be sure to update all pipelines configurations in Concourse. In this way I can achieve GitOps for pipelines.

@kwladyka I think you can achieve your use case even today, by specifying a single Job with the Sync hook annotation, and no "normal" application resources.

[jessesuen]

Another important point for users interested in this feature, is that if you are using kustomize to manage configs, kustomize does not support generateName well. See:

https://github.com/kubernetes-sigs/kustomize/issues/586

[wmedlar]

kustomize does not support generateName well

I've been able to workaround this behavior, at least in Kustomize v1, by patching in generateName with the patchesJson6902 field:

# kustomization.yaml
resources:
- job.yaml

patchesJson6902:
- path: patches/job-generate-name.yaml
  target:
    group: batch
    version: v1
    kind: Job
    name: foo

# job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: foo
spec: ...

# patches/job-generate-name.yaml
- op: move
  from: /metadata/name
  path: /metadata/generateName

and finally the compiled manifests:

$ kustomize build
apiVersion: batch/v1
kind: Job
metadata:
  generateName: foo
spec: ...

Works like a charm so long as you don't try to modify the Job spec after the patch.

[jessesuen]

Great tip! I'm going to reference your workaround in the original kustomize bug I filed

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[so0k]

as far as I can tell, ArgoCD supports jobs with generateName only if they have the special annotation, which tells Argo not to use kubectl apply but kubectl create instead and it seems the only way to add jobs as part of your application

[lallinger-arbeit]

For anybody stumbling across this and wondering which annotation you have to set refere to this: https://argoproj.github.io/argo-cd/user-guide/resource_hooks/