Argo-cd: How to sync local changes with argocd

[narg95]

Hi all,

I would like to know what is your recommended way to setup an app with argocd for daily development, that is to sync local changes to a local kubernetes cluster e.g. to minikube. By local changes I mean, commited and uncommited changes made in a local branch to the app resource manifests as well as to the app source-code.

I took a look to the documentation but I could not find a clear way to do it, I only found that there is a way to sync an app with a local file using flag --file, but how to generate this file is not clear, also if the file is watched for changes, and if that is the way to do it for local development.

My scenario could be something like this:

• There are two repos: i)app source-code and ii) app resource manifests
• I have cloned both in my local/dev machine
• I have created a feature branch for both
• I made some local changes to both, app source-code and app resource manifests
• I want those changes to be sync into minikube for testing
• I build the Docker images with the changes to the app source-code, and they are available to be pull inside minikube (e.g. using eval $(minikube docker-env))

so, how to sync those changes into minikube without much hassle? :)

Thanks in advance!

[jessesuen]

HI @narg95, without making commits to the git config repo, you can use the parameter override feature to set the image or image tag. See: https://github.com/argoproj/argo-cd/blob/master/docs/parameters.md.

Parameters require the use of templating tools that support parameters. We currently support ksonnet and helm parameters. In v0.11, we will support kustomize's imagetags as parameters. The idea is that instead of pushing temporary imagetag changes into git when you are doing local development, you can run:

argocd app set guestbook -p image=example/guestbook:abcd123
argocd app sync guestbook # optional if you have auto-sync

These parameters will override any parameters defined in git.

You might also want to check out https://github.com/GoogleContainerTools/skaffold, which is designed to solve your use case.

On a personal note, I don't really bother with Argo CD or skaffold for local development, since I just use the :latest tag and delete the pod and let it restart after pushing out new changes.

[narg95]

Hi @jessesuen, thanks for the quick response!

You might also want to check out https://github.com/GoogleContainerTools/skaffold

Nice hint, it just feels like javascript and webpack hot-reload ;)

I just use the :latest tag and delete the pod and let it restart after pushing out new changes.

Right, it is very handy and what I will do, but what-if I change multiple entries in ConfigMaps? updating using argocd app guestbook -p is error-prone and less declarative. Since I will have minikube as a environment in the resource manifest repo:

• Is there any way for argocd to take local-manifest changes and sync them to minikube? it should not be automatic, e.g. in the create subcommand

argocd app create guestbook-minikube \
--local-repo ./argocd-example-apps/ \
--path ksonnet-guestbook \
--env minikube

Actually, that could be done by setting up a local git server, but I would like to avoid that step.

• or would you use in this case the templating tool for deploying to minikube e.g. ks apply minikube or kustomize build ~/gestbook/overlays/minikube | kubectl apply -f -?

[jessesuen]

Is there any way for argocd to take local-manifest changes and sync them to minikube? it should not be automatic, e.g. in the create subcommand

This is not possible with Argo CD, since manifests are always fetched/pulled from a git repo, and the use of Argo CD requires the use of a git server.

or would you use in this case the templating tool for deploying to minikube e.g. ks apply minikube or kustomize build ~/gestbook/overlays/minikube | kubectl apply -f -?

Yes, that is what I would do. However, this approach will not get the injected labels that Argo CD currently depends on in v0.10, so the UI will not be able to render the resource tree. However it will work better in v0.11, when we rely less on application labeling in order to render the resource tree.

[jessesuen]

Had some more discussion about this. We do agree it would be useful to have a feature where argocd app sync APPNAME can accept a --local <PATH> to deploy manifests from a local directory instead of from git. This would make the dev & test use case much easier.

The way this could work is that argocd app sync --local <path> would first locally run the templating command against the local path to generate manifests, then supply the entire set of YAML as part of the sync request. When the application controller reconciles the application, it would skip the git repo-server query (since it already has the set of manifests), and deploy the inlined manifests directly.

There are some security concerns with this. Currently, a SyncOperation can only deploy manifests from the application's git repo. This makes it safer to give "sync" permissions to a CI pipeline because even if token is compromised, the attacker would need write access to the git repo, to deploy malicious manifests.

For this feature, some additional RBAC requirements would need to check that the caller has both app "update" and "sync" permissions.

Another consideration is that rollback would not be supported, and we would not record a local sync to the sync history.

[narg95]

Sounds great!

I workaround it so far by serving my local repo with git daemon, the only issue is that I have to commit my changes, or amend the last draft commit.

Some questions/thoughts about it:

• what would happen if I use --sync-policy automated? would it watch for file changes and apply them?
• why not adding the --local <PATH> intead to the app create command? I think the--local <PATH> has global implications to other commands like argocd get and argocd diff? I mention it because in my current workarounded workflow I usually:
  
  
  
  1. change some resource manifests
  
  
  2. check what is out-of-sync (using argocd app get my-app --refresh)
  
  
  3. sometimes I see the diff (argocd app diff my-app)
  
  
  4. and I apply the changes to my local cluster (argocd app sync my-app)
  
  

Perhaps, thinking further, a feature e.g. like argocd app watch --show-diff --interactive or something alike could automate this and make the dev experience even better.

So for example, I create an app with argocd app create my-app --local <PATH>, and then I watch:

$ argocd app watch my-app
Service              my-service         OutOfSync
Deployment           my-deployment      OutOfSync
[changes applied ]

with --show-diff

$ argocd app watch my-app --show-diff
Service              my-service         OutOfSync
Deployment           my-deployment      OutOfSync

===== my-app/my-service ======
0a1,172
> apiVersion: apps/v1beta2
> kind: Deployment
> metadata:
>   annotations:
...

with --interactive you controll when to apply the changes to prevent many pod restars. If new changes come then it refreshes and ask again:

$ argocd app watch my-app  --interactive

Service              my-service         OutOfSync
Deployment           my-deployment      OutOfSync

apply changes (y/n) ?

Thanks for reading !
P.S. great job with argocd!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alexec]

@simster7 is this addressed by your --local option do you think?

[alexec]

Maybe fixed by #1578

[simster7]

Yup, this is what caused #1578 (now merged)

[alexec]

🚀 closing.

[alexec]

Regression comments:

• Works as expected with one minor bug.