Carthage: Parallel building

[jeeftor]

I know there are some flags (which I'm not 100% sure which they are however) that allow Xcode to do something similar to a make -j4 parallel style build.

Does carthage support these flags and if so what are they, and if not can we get support for parallel building? The slowest part of my CI tasks is the Carthage building.

Thanks

[mdiep]

Carthage doesn't do any parallelization, but the individual projects that it compiles may have parallelized builds turned on in Xcode. (Those would be respected, as carthage calls through to xcodebuild.)

It would certainly be possible to add this to Carthage directly. But I'm not sure how much sense it would make because it would most likely lead to contention for CPU/RAM/disk. (I used to have a Mac Pro where disabling some of the CPU cores would lead to faster compilation because of RAM contention.)

If you're looking to speed up CI tasks, you're better off using binary builds or something like https://github.com/guidomb/carthage_cache.

[jeeftor]

Our issues is we exist inside a firewall and are not allowed to use github so binaries are no dice

Sent from my iPhone

On Feb 8, 2016, at 10:17 AM, Matt Diephouse [email protected] wrote:

Carthage doesn't do any parallelization, but the individual projects that it compiles may have parallelized builds turned on in Xcode. (Those would be respected, as carthage calls through to xcodebuild.)

It would certainly be possible to add this to Carthage directly. But I'm not sure how much sense it would make because it would most likely lead to contention for CPU/RAM/disk. (I used to have a Mac Pro where disabling some of the CPU cores would lead to faster compilation because of RAM contention.)

If you're looking to speed up CI tasks, you're better off using binary builds or something like https://github.com/guidomb/carthage_cache.

—
Reply to this email directly or view it on GitHub.

[bwhiteley]

I would like to see Carthage build multiple frameworks concurrently, when possible. It currently takes 25 minutes to build our dependencies with Debug configuration, and cores are nearly idle for much of that time.

[peterromfeldhk]

if the CI is running different jobs in parallel that have same deps i'm running to issue index.lock: File exists.

carthage bootstrap --no-use-binaries --platform iOS --derived-data $(mktemp -d) --cache-builds --log-path fastlane/build_output/carthage.log

[ThePragmaticArt]

@peterromfeldhk This has been an issue forever. I'm frankly baffled that anyone expects Carthage to scale if this functionality is not baked in. As soon as framework bundles are available via SPM, we're doing a full migration (rumor is framework bundles are currently in the works per some birdies at Apple). SPM's performance is just light years ahead of Carthage and far more scalable from a caching perspective considering the majority of the cache is assigned in the individual project's derived data. I would argue that the lack of this functionality nearly cripples Carthage for CI/CD without extra work going into binary deployments (which isn't always desirable in an enterprise environment where you own the code and are rapidly changing it).

The closest solution we have is the lovely nsoperation/carthage fork generously supported by another dev. This project has single handedly displayed how absurdly inefficient Carthage is at this point in time (3 years later...). Unfortunately, as of now, it has issues running against the new Swift headers in Xcode 11. My company has practically swapped to using this out of desperation to migrate away from the primary Carthage branch without migrating to something like Submodules or CocoPods.

Some people literally don't understand how important parallel tasks are to scaling...

[tmspzz]

@ThePragmaticArt feel free to suggest and implementation and submit a PR

[unchartedworks]

If anyone wants to see Carthage build multiple frameworks concurrently, when possible. Please check it out. https://github.com/unchartedworks/ConcurrentCarthage