Carthage: Segmentation fault when running `carthage bootstrap`

[vytis]

• carthage install method: [ ] .pkg, [x] homebrew, [] source
• which carthage: /usr/local/bin/carthage
• carthage version: 0.28
• xcodebuild -version: Xcode 9.0.1
• Are you using --no-build? No
• Are you using --no-use-binaries? No
• Are you using --use-submodules? No
• Are you using --cache-builds? No
• Are you using --new-resolver? Yes

Cartfile.resolved

github "wireapp/AFNetworking" "2.6.3-iOS8"
github "wireapp/Cartography" "3.0.1"
github "wireapp/Classy" "v0.2.6-swift3.0.2"
github "wireapp/CocoaLumberjack" "3.3.0"
github "wireapp/Down" "v1.0.2-test"
github "wireapp/FLAnimatedImage" "1.0.12-wire"
github "wireapp/FormatterKit" "1.8.1-swift3.0.2"
github "wireapp/HockeySDK-iOS" "5.0.0"
github "wireapp/PINCache" "2.3-swift3.1"
github "wireapp/PureLayout" "v3.0.0"
github "wireapp/TTTAttributedLabel" "2.0.0-swift3.0.2"
github "wireapp/ZipArchive" "v2.0.8"
github "wireapp/avs-ios-binaries" "4.1.131"
github "wireapp/ios-snapshot-test-case" "2.1.4-swift3.1"
github "wireapp/libPhoneNumber-iOS" "0.9.2-swift3.1"
github "wireapp/mixpanel-swift" "v2.2.2"
github "wireapp/ocmock" "v3.4"
github "wireapp/onepassword-app-extension" "framework/1.8.2-wire-swift3.0.2"
github "wireapp/ono" "1.4.0"
github "wireapp/protobuf-objc" "1.9.14"
github "wireapp/wire-ios-canvas" "5.0.2"
github "wireapp/wire-ios-cryptobox" "11.0.1"
github "wireapp/wire-ios-data-model" "93.2.2"
github "wireapp/wire-ios-images" "14.0.0"
github "wireapp/wire-ios-link-preview" "7.0.1"
github "wireapp/wire-ios-message-strategy" "79.0.0"
github "wireapp/wire-ios-protos" "13.0.0"
github "wireapp/wire-ios-request-strategy" "77.0.0"
github "wireapp/wire-ios-share-engine" "66.0.0"
github "wireapp/wire-ios-sync-engine" "149.1.0"
github "wireapp/wire-ios-system" "19.1.0"
github "wireapp/wire-ios-testing" "13.0.1"
github "wireapp/wire-ios-transport" "34.0.2"
github "wireapp/wire-ios-utilities" "20.2.0"
github "wireapp/wire-ios-ziphy" "7.0.1"

Carthage Output

*** Downloading PureLayout.framework binary at "v3.0.0"
*** Downloading ocmock.framework binary at "v3.4"
*** Downloading wire-ios-utilities.framework binary at "20.2.0: It smells like a hesitant freedom"
*** Downloading mixpanel-swift.framework binary at "v2.2.2"
*** Downloading AFNetworking.framework binary at "2.6.3-iOS8"
*** Downloading wire-ios-message-strategy.framework binary at "79.0.0: cicadas really love eating feta"
*** Downloading Cartography.framework binary at "3.0.1"
*** Downloading wire-ios-transport.framework binary at "34.0.2: Woodpecker finch"
*** Downloading wire-ios-canvas.framework binary at "5.0.2: MODOK VS. Owlman"
*** Downloading avs-ios-binaries.framework binary at "Version 4.1.131"
*** Downloading wire-ios-share-engine.framework binary at "66.0.0: I'd like a calm salad greens and cheddar burger without chia seeds, please"
*** Downloading ZipArchive.framework binary at "v2.0.8"
*** Downloading wire-ios-testing.framework binary at "13.0.1: That time when the compiler said 'member' is not a valid template type member"
*** Downloading wire-ios-system.framework binary at "19.1.0: Kyllikki the functional orog from Homberg"
*** Downloading protobuf-objc.framework binary at "macOS"
*** Downloading wire-ios-images.framework binary at "14.0.0: The One with the Yeti"
*** Downloading wire-ios-request-strategy.framework binary at "77.0.0: A giant trite horse was spotted near Lenzen"
*** Downloading wire-ios-protos.framework binary at "13.0.0: Please estimate the weary otter"
*** Downloading ono.framework binary at "Swift 3.1"
*** Downloading wire-ios-link-preview.framework binary at "7.0.1: The intermediate radical frog"
*** Downloading wire-ios-ziphy.framework binary at "7.0.1: From Nanning to Papeete"
*** Downloading PINCache.framework binary at "Swift 3.1"
*** Downloading HockeySDK-iOS.framework binary at "5.0.0"
*** Downloading FLAnimatedImage.framework binary at "1.0.12-wire"
*** Downloading ios-snapshot-test-case.framework binary at "Swift 4.0"
*** Downloading libPhoneNumber-iOS.framework binary at "Swift 3.1"
*** Downloading FormatterKit.framework binary at "Swift 3.0.2"
*** Downloading Down.framework binary at "Libcmark liberation"
*** Downloading onepassword-app-extension.framework binary at "Swift 3.0.2"
*** Downloading wire-ios-sync-engine.framework binary at "149.1.0: Leverkusen"
*** Downloading TTTAttributedLabel.framework binary at "Swift 3.0.2"
*** Downloading Classy.framework binary at "Swift 3.0.2"
*** Downloading CocoaLumberjack.framework binary at "3.3.0"
*** Downloading wire-ios-cryptobox.framework binary at "11.0.1: You should never harm in Huai'an"
*** Downloading wire-ios-data-model.framework binary at "93.2.2: What would Mary Lou Jepsen do?"
*** xcodebuild output can be found in /var/folders/5g/75n6zq454r3dfyh0g_cm9m_h0000gp/T/carthage-xcodebuild.twJkJh.log
carthage(71417,0x7fffb3a603c0) malloc: *** error for object 0x10f751d08: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
./setup.sh: line 46: 71417 Abort trap: 6           carthage bootstrap --platform ios

Actual outcome
Carthage crashed with segmentation fault

Expected outcome
Carthage should not crash

---

Our project is open source (repo) and can be checked out easily.
We have been seeing a lot crashes running carthage bootstrap --platform ios on CI machines, both local and on CircleCI (1, 2). It is happening almost daily, so I think there is something special with contents and/or size of our Cartfile.

I have managed to reproduce this (or maybe other?) crash with latest master (https://github.com/Carthage/Carthage/commit/192a61d37b6ad27ec5d20d0d267ea3e70917689a) on my local machine from Xcode.

I am running carthage with bootstrap --platform iOS --new-resolver arguments and pointing the directory to where wire-ios is checked out (using instructions from https://github.com/Carthage/Carthage/blob/master/CONTRIBUTING.md). Note: we are still on Xcode 9.0.2, dependencies will be recompiled if you use a different Xcode version.

It seems to be crashing with EXC_BAD_ACCESS here:

I don't know much about ReactiveSwift or Carthage codebase to debug/fix it myself, but I can reproduce this every 3-5th run.

I hope this helps to shed some more light into the problem as it seems to have been reported a few times, however hard to say if it is actually the same reason behind all of them https://github.com/Carthage/Carthage/issues/2320

[mdiep]

That's very helpful! Thank you. That should help us investigate the issue further.

[vytis]

Great! Let me know if you need any more information about our current setup.

[tmspzz]

I'm getting a crash even earlier with 100% repro.

Happens when running bootstrap with no Cartfile.resolved

Xcode9.2

[tmspzz]

@vytis I have run carthage bootstrap --platform iOS --new-resolver 10 times in a row with the Cartfile, Cartfile.resolved and Cartfile.private in Wire's repo and can't repro with Xcode9.2 :(

[vytis]

Maybe trying out carthage update instead of bootstrap would make it crash again? I had luck after 5-6 tries today again, but doing an update. It seems to always be happening when downloading binaries. We still use Xcode 9.0.2, so anything else than that will force you to recompile everything and maybe that makes a difference...

[vytis]

I ran it with Thread Analyzer turned on and it caught the issue on the same line of code immediately. Seems to be that platformCaches dict is being modified from two different threads:

[tmspzz]

@vytis as I suggested via other channels (don't know if it got to you), try wrapping all read and write access in a serial queue

[mdiep]

Instead of using .on(value:) to modify a local variable, that code should be modified to build up the VersionFile with SignalProducer.reduce.

[vytis]

I will try to make a PR with how I understood what you have in mind and maybe we will have something tangible to discuss about! :)

[tmspzz]

Fixed in #2376