Tilt: Run builds in parallel

[ivansantosz]

Would that be reasonable to have an option for docker_build in parallel? Looks like when having multiple microservices, they are following a sequential build.

[nicks]

Yep, tilt currently does a sequential build.

Why do you want them to happen in parallel? Do you have an idea of which builds you would mark as parallelizable and which ones you wouldn't?

We've talked about this a little. The downside it's also much harder to understand and diagnose problems when multiple builds are going on at the same time. One idea I've had is to do the cpu-intensive parts of the builds sequentially, but pre-pull the FROM images in parallel.

[ivansantosz]

As far as I understand, builds don't necessarily need to be related, the only benefit of having that as a sequence is to reuse some layers of other services as a cache. Which may not be the case for different projects where microservices use different langs/services.

The UI seems to be very in sync with having builds in parallel. It already happens for kubernetes yml apply.

[fkaleo]

This sequential building makes docker-compose currently faster than tilt at startup.

[ajbouh]

In the same boat. On a machine with many cores, building in parallel makes a huge difference.

In my experience skaffold also starts much more quickly than tilt.

[drubin]

Yes, same boat here, all our "builds" should be able to be run in parallel but rolling them out sequentially makes more sense kinda?

I feel like if we are trying to promote k8s for dev we need to also take into account that in production these rollouts won't be sequential and it would help track down interdependencies.

We have a service with lots of mini services tiny ones but the building them takes time and doing them so the sequential part of tilt would really impact performance for our one team.

[tmc]

Often image builds are not maximizing machine resources and have io-bound portions (especially true with lots of python services). docker-compose's parallel building has the downside of mixing all the output -- this could be a way that tilt stands out even further as a better experience.

[nicks]

This is launched as of v0.11.0 !