Skaffold: Support remote helm chart repositories

[nikParasyr]

Hello. I tried using skaffold with a remote chart repository. The reason behind it is that we have different repos for our "app code" and our "infra code" (mostly helm charts). To my understanding this functionality is not supported.

My test-case was to modify the examples/helm-deployment to deploy to minikube with a chart from the local helm repo

local http://127.0.0.1:8879/charts

(also tried various other scenarios on gke and the stable chart repo, etc, same error occur)

The error is that helm dep build seeks for the requirements.lock file locally which is not the case here, and there is no magic flag to support building dependencies from a remote chart(or at least i haven't found any while reading helm docs). A workaround, in the case of a remote chart repo which could be defined in skaffold.yaml, might be to fetch and unpack the chart locally -> install/upgrade -> clean-up but it seems kind of an overkill.

Expected behavior

Run skaffold using the remote helm chart

Actual behavior

Skaffold dev/run fails with output:
WARN[0000] run: deploy step: deploying skaffold-helm: helm dep build stdout: , stderr: Error: could not find local/skaffold-helm: stat local/skaffold-helm: no such file or directory : exit status 1

Information

Skaffold version: v0.2.0
Operating system: Linux 4.13.0-36-generic #40~16.04.1-Ubuntu
Content of skaffold.yaml: 
 apiVersion: skaffold/v1alpha1
 kind: Config
 build:
   tagPolicy: sha256
   artifacts:
   - imageName: skaffold-helm
     workspace: .
   local: {}
 deploy:
  helm:
     releases:
     - name: skaffold-helm
       chartPath: local/skaffold-helm
       namespace: default
       values:
         image: skaffold-helm

Steps to reproduce the behavior

1. Package and serve the skaffold-helm chart from local helm repo: helm package skaffold-helm; helm serve
2. Change skaffold.yaml chartPath to local/skaffold-helm
3. skaffold dev

[r2d4]

Interesting use case, thanks for bearing with us as we start to explore and support some of these workflows. I've seen this workflow in practice before. Can you just verify what your app repo vs. infra repo looks like?

app repo:
values.yaml?
app code
Dockerfile

infra repo:
charts
cluster-level kubernetes manifests (storage classes, CRDs, etc.)?

Do you use one umbrella chart for all of your apps or does each app have its own chart?

I think we should support this workflow of remote deploy objects, but local code and build objects. I'm not sure the best path forward for making this work with helm, for the kubectl deployer its a bit simpler (kubectl can accept remote manifests).

cc @dlorenc @viglesiasce

[nikParasyr]

Every app has its own chart.

app repo:
app code
Dockerfile

infra repo:
Charts (deploy,svc,pvc,etc)

Cluster-level kubernetes manifests are at seperate repos.

Thanks for all the work btw, really nice tool.

[faheem-nadeem]

Hi. We have a pretty similar workflow. Charts are placed in a separate git repository and are being served automatically by chartmuseum at multiple endpoints https://kubernetes-charts.example.com/org/<stable/incubator>. Chart repository usually has some depth to it. i.e. org/<stable/incubator>/<chart-name>. These repositories are then usually added to a dev local helm client with helm repo add mystablerepo https://kubernetes-charts.example.com/org/stable. Hence installing a chart is as simple as e.g. helm install --name mycassandra --version a.b.c --values values.yaml mystablerepo/cassandra. The values.yaml file is usually kept in the app repository together with the Dockerfile in a deploy folder and/or can be sane defaults from the charts repository values.yaml file. We also use separate yaml files for dev, staging, productions workflows. Currently we bundle all of this using Makefiles in app repo. It would be awesome if remote chart repositories are supported in Skaffold to ease a lot of pain while development :)

[upalkhouski]

Hello guys. We are considering using skaffold for similar case - keep skaffold.yaml with definitions of all the plugins (remote helm charts) and set of addon-value.yaml files to manage values of each addon (chart). I suppose the intention was/is to introduce remote charts support based on the following comment:
https://github.com/GoogleContainerTools/skaffold/blob/daefae119cdb44fdf3286a60b7dbd08931262603/pkg/skaffold/deploy/helm.go#L147

[nathkn]

Would a potential solution be to simply skip the helm dep build <repo> step for remote charts from repositories?
https://github.com/GoogleContainerTools/skaffold/blob/daefae119cdb44fdf3286a60b7dbd08931262603/pkg/skaffold/deploy/helm.go#L136

Similar to how I might run a command such as helm install --dep-up stable/mariadb ..., which would rely on helm install's built-in pulling of dependencies, and on the chart owner to update the requirements appropriately.

The benefit of using skaffold would then be the ability to pass in build artifacts to that remote chart with --set as well as a reproducible deploy pipeline.

[faheem-nadeem]

Any luck getting this fixed, or is there support already? :)