Choco: Parallel download

[graingert]

Option to download all packages and dependencies in parallel before executing the install phase

[graingert]

passing a batch of URLs to https://chocolatey.org/packages/aria2 would also solve https://github.com/chocolatey/choco/issues/402

[graingert]

this would also allow pro users to download from the CDN and the upstream source simultaneously

[ferventcoder]

Howdy! This is an interesting request. I know it's something that Yum did/does when you ask for a package with dependencies. It goes an gets everything first after verification and then once it has everything it needs, it gets started.

I'd love to be able to handle this, but I'll also state that it would take a complete reengineering and owning of a lot of the work that NuGet currently does. So not out of the realm of possibilities, but definitely not in the short term range.

[kfsone]

chocolatey is currently agonizingly slow compared to other package managers, and when operating on desktop I frequently find myself questioning whether it wouldn't just be faster to do this by hand. I honestly just spent a half hour reading up on how to use ansible for a windows host; the things that happen when you drink decaf.

The issue is that choco is so bloody serial.

I think a better ticket than parallel download would be re-orchestrating the entire flow of chocolatey to use a conveyor belt approach and be concurrent. Do all the package lookups up-front and then start dispatching them for download. As they come back from downloading, install them (serially) as soon as dependencies are met.

This alone would drastically speed up chocolatey.

[vexx32]

Tbh even if we just look at each requested package separately and then download them with dependencies, each main package on a separate thread, we'd probably get a pretty decent speedup.

[eladkarako]

see how https://github.com/ilikenwf/apt-fast wraps apt-get install ..., and do the same...

for a long time, my setup includes on its very start:

add-apt-repository ppa:apt-fast/stable
apt-get update
apt-get -y install aria2 apt-fast

apt-fast ..........

I also choose --split=3 --max-concurrent-downloads=32 --max-connection-per-server=16 --min-split-size=1M