Argo: What's your favorite CI use-case around Worflows ?

[sbose78]

Greetings Argo Worklows community 👋 !

This could be an existing use case or something you wish was present.

Proposed response template ( not exhaustive ):

• Describe the use case/scenario ( could be an essay, that's fine! :) )
• Has the use case been addressed in other traditional CI systems ?
• Is the use case being addressed in other kubernetes-native CI projects ?

Starting this thread so the user community can contribute use cases which at the least should lead to community-managed well-documented scenarios/solutions 😺

[adnankobir]

Most CI tools today are not kubernetes native and therefore do not take advantage of many out-of-the-box features; for instance, one use-case would be to run CI steps with different permissions. Argo workflows today allows you to run steps with different serviceAccounts which can be tied to RBAC as well as IAM permissions via IRSA.

As an example, I would love to use Workflows to build a docker image, publishing to ECR, upload assets to S3, deploying a helm chart etc. The advantage here is that each of these steps can have fine-grained controls in place via RBAC and IRSA and we can avoid creating a blanket serviceaccount/role with overreaching permissions.

Scaling workloads is another example where Workflows would shine as a CI tool, the k8s scheduler and cluster autoscaler would already provide the basics, but we can take this a step further by leveraging prometheus metrics from Workflows and expose them as custom metrics to be consumed by HPA and scale an external microservice that might need to ingest the output of the workflow.

Although this can be achieved by another kubernetes-native tool like tekton, the self-service approach to Workflows and the UI (not to mention the maturity of Workflows) make the barrier to entry much lower than other projects

[dcherman]

Adding on to what @adnankobir said (which I 100% agree with), I see enough similarities between Tekton and Argo that I would question whether or not it makes sense to install (and learn how to maintain/upgrade) both systems. Since CI is a specific subset of workflow orchestration, it seems like we could potentially solve this with Argo and reduce the number of projects that need to be understood/maintained.

[adnankobir]

Another aspect to note here: it would be nice if Workflows can incorporate triggers/Argo Events more seamlessly, as it stands today I feel like the separation of triggers/events from Workflows makes it difficult to implement gitops. For instance, many CI tools today will automatically inject webhooks into your desired repository and have semantics for executing pipelines based on github events. Although I suppose this can be automated (another workflow?), it would be nice if we had some sort of first-class support for this directly in Workflows.

[alexec]

It would be good to understand the reason people choose Argo Workflows for CI rather than other solutions.

[b2cbre]

Our team is struggling to determine how one may use Argo Workflows for CI type work, like running tests on a specific commit whenever a PR is opened in GitHub. So this GH Issue is very interesting to us.

[adnankobir]

It would be good to understand the reason people choose Argo Workflows for CI rather than other solutions.

@alexec the reasons vary for people's use-cases, personally my primary reason to choose Argo Workflows is native support for kubernetes: other solutions will usually do things like docker-in-docker to launch workflow steps (which would require privileged access), incorporate execution steps into a single binary (suffers from scaling), lack proper AuthZ/no support for RBAC and in a lot of cases implement their own authZ, have very opinionated ways to pass artefacts or end up having a global shared state for workers (for ex. jenkins plugin ecosystem - which makes it extremely unmanageable for large teams that might have conflicting versions/dependencies). Secrets are another huge pain point, we use vault extensively and want to take advantage of the native kubernetes Auth it provides, but to-date, I've not come across a single ci solution that integrates with it (simply because they don't supporting providing servicesAccount per workflow execution step as I've stated in my first comment).

I feel like the Argo Workflow project addresses all these pain points extremely well and just requires a bit of work to address or bridge the gap between Workflow and Events to make a truly robust and powerful CI tool.

A perfect example of bridging the gap between Workflow and Events is identified here by @b2cbre

Our team is struggling to determine how one may use Argo Workflows for CI type work, like running tests on a specific commit whenever a PR is opened in GitHub. So this GH Issue is very interesting to us.

[torstenwalter]

https://github.com/jenkins-x/lighthouse is built to trigger Tekton pipelines if I understand it correctly.

Maybe it could be enhanced to also support Argo workflows or at least provide building blocks for it.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.