Skaffold: Better handling for multiple microservices (ability to handle multiple skaffold.yaml files)

[ahmetb]

I currently have a microservices app that has 3 parts. So I have to run these concurrently in separate shell tabs:

skaffold dev -f file1.yaml
skaffold dev -f file2.yaml
skaffold dev -f file3.yaml

I have to do this every time I launch my workspace.

Now, if there was a good unix tool (like GNU parallel) that can start, stream output of, and manage lifecycle of multiple processes at once, that'd be awesome, but it doesn't exist. So we must consider adding skaffold ability to accept multiple files like:

 skaffold dev -f file1.yaml file2.yaml file3.yaml

which can be written easily in bash like

 skaffold dev -f file{1,2,3}.yaml

---

The problem is not to skaffold dev and skaffold run is suffered by this too.

For example my cloudbuild.yaml (Google Cloud Container Builder) contains a lot of repetition, whereas it could've been a single step with multiple files (which may even be executed in parallel within skaffold command)

steps:
- name: gcr.io/ahmetb-public/skaffold
  args: ['run', '-f=file1.yaml']
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=coffee'
steps:
- name: gcr.io/ahmetb-public/skaffold
  args: ['run', '-f=file2.yaml']
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=coffee'
steps:
- name: gcr.io/ahmetb-public/skaffold
  args: ['run', '-f=file3.yaml']
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=coffee'

[d11wtq]

I want this too, but I think something like forego or foreman should suffice to manage multiple skaffold dev commands together.

[sniederm]

I would also love to see such a feature in Skaffold. Right now I have 5 different microservices each running as a single pod/service. I want to re-build and re-deploy only the service those source code has changed. So I have to keep open 5 terminals in dev mode. Would be great if we could have Skaffold to monitor multiple pipelines with multiple [image + deployment] sets within one process or monitoring multiple files as @ahmetb suggested.

[VVKot]

@ahmetb @d11wtq
I have a setup like so - say, 3 microservices serviceA, serviceB, serviceC in separate repos and another repo with ESB.

If I want to change serviceA, is it correct that the best I can do with skaffold at the moment is like so?

skaffold dev -f serviceA.yaml
skaffold run -f serviceB.yaml
skaffold run -f serviceC.yaml
skaffold run -f esb.yaml

[tstromberg]

I believe that @balopat has been recently thinking about how to best implement this, though I'm not sure he had multiple YAML files in mind.

[balopat]

@VVKot can you have a look at this design doc: http://tinyurl.com/skaffold-modules? I would love to have your opinion.

[smhmayboudi]

@ahmetb do you have any dependency which shares between your services? if yes, how do you manage them? I have two microservices which I add the reddis as a dependency to both helm charts and I am trying to run then with one skaffold file. However when the second one goes up, it deletes the first one !?

[smhmayboudi]

Deploy multiple microservice is not working with helm in the case which chart needs the same dependency. skaffold changes the meta types of the 1st microservice dependency and the second microservice with the same dependency wants to update and it is not working.
1st dep app.kubernetes.io/managed-by=skaffold-v...
2nd dep app.kubernetes.io/managed-by=Helm

[nkubala]

@smhmayboudi I believe your use case will be solved by the modules feature which we're actively working on - this should allow you to specify dependencies between artifacts, and have rebuilds and redeploys work as intended. check out the design doc linked in the above comment for details, and stay tuned for updates on that feature.

[ItsLeeOwen]

if skaffold dev supported specifying services to build/deploy defined within the yaml, similar to docker-compose up that'd be really useful.

skaffold dev servicea serviceb servicec