Flux: Mutli k8s Cluster deployment support from single git repository

[yashghia]

Describe the feature
Introduce capability in flux to support syncing multiple clusters using single shared git repository.
i.e The cluster runs same application loads but with different configurations.
If Flux can have ability to monitor subfolders in repo (configuration folder), we could have a starting point to support multiple cluster.

What would the new user story look like?
Given we have multiple k8s clusters to manage using same repo, when a user provides configuration path then the Flux on cluster is setup such that it observes the single shared git repository for any changes specific for own cluster and sync

Expected behavior
Individual K8s cluster should be able to observe the git repo and sync if there is any configuration changes for it.

[c4m4]

--git-path option can help you, in your shared repo you could have a directory per cluster

[yashghia]

Appreciate your response. I understand the usage of git-path but that doesn't exactly tackle my problem. I will try and explain the problem using an example git folder structure

repo

• Helm
  
  
  
  • App1
    
    Deployment.yaml
    
    ConfigMap.yaml
  
  
  • App2
    
    Deployment.yaml
    
    ConfigMap.yaml
    
    ....
    
    ....
  
  
• Config_Folder
  - Cluster1
  - App1
  config.yaml
  - App2
  config.yaml
  - Cluster2
  - App1
  config.yaml

Now what I wish is, flux having some kind of configurable setting set to config folder path which needs to be applied to particular cluster while deploying application. In above example, Cluster1 flux is configured to look for changes in App1 folder and Config_Folder/Cluster1/

[stefanprodan]

This is exactly what git-path does, you can give it one or more paths and it will sync only those on the target cluster.

[nshampur]

@stefanprodan - what we were looking for was to have different Values.yaml file per environment (e.g:- Prd | Staging | Dev) while using the same helm charts and folder structure.
How would we achieve that with a single repo and the Helm operator? Should we git-path exclude/include the appropriate values.xaml file?
I guess the answer lies in - digging into how the helm operator finds and uses Values.yaml files to apply for its charts.

[stefanprodan]

Create a HelmRelease per env and use the spec.values to override some values from the values.yaml. Helm Operator merges values.yaml with spec.values.

[nshampur]

@stefanprodan - thanks for the speedy response. While that approach works - we were hoping for something more scalable when we have a large number of configuration keys that are environment (cluster) specific. This would include non secret information and secret.

Ideally spec.values could reference a collection of paths that the helm operator would use to apply during its execution.

[stefanprodan]

You can refer secrets and configmaps in a HelmRelease please see the docs https://docs.fluxcd.io/projects/helm-operator/en/stable/helmrelease-guide/values/#values

[yashghia]

@stefanprodan Thanks for directing us to appropriate doc. Gave it a try with either values from externalSourceRef or chartFileRef with single or multiple Values.yaml files and they work.