Carthage crashes with Segmentation Fault: 11

[JetForMe]

Carthage crashes with Segmentation Fault: 11, but running it again usually succeeds. It's not clear to me what changes to cause it to happen again.

• carthage install method: [ ] .pkg, [X] homebrew, [ ] source
• which carthage: /usr/local/bin/carthage
• carthage version: 0.33.0
• xcodebuild -version: Xcode 10.1
  Build version 10B61, but also happens with 10.2
• Are you using --no-build? no
• Are you using --no-use-binaries? no
• Are you using --use-submodules? no
• Are you using --cache-builds? no
• Are you using --new-resolver? no

Cartfile

github "agilebits/onepassword-app-extension" "add-framework-support"
github "Alamofire/Alamofire" ~> 4.0
github "dzenbot/DZNEmptyDataSet" ~> 1.8.1
github "m1entus/MZFormSheetPresentationController" "master"
github "mxcl/PromiseKit" ~> 6.0
github "PromiseKit/PMKAlamofire" ~> 3.2
github "segmentio/analytics-ios" ~> 3.6
github "square/Valet"
github "latencyzero/Marshal" "escapePeriods"

Carthage Output

$ carthage update --platform ios
*** Fetching Valet
*** Fetching Marshal
*** Fetching MZFormSheetPresentationController
*** Fetching onepassword-app-extension
*** Fetching DZNEmptyDataSet
*** Fetching analytics-ios
*** Fetching PromiseKit
*** Fetching Alamofire
*** Fetching PMKAlamofire
*** Fetching MZAppearance
*** Checking out MZAppearance at "de07b5fdd79465e4193095cd33a66843106e71de"
*** Checking out onepassword-app-extension at "bcc4cc97fed9a6e73fa204f2e61138e353cb3ef7"
*** Checking out MZFormSheetPresentationController at "6f4b7c8875baaa1a881c889934c7b9901155d393"
*** Checking out analytics-ios at "3.6.10"
*** Checking out DZNEmptyDataSet at "v1.8.1"
*** Checking out PromiseKit at "6.8.4"
*** Checking out Valet at "3.2.3"
*** Checking out Alamofire at "4.8.2"
*** Checking out Marshal at "cd203eb665d1ec0e1c04f26537316365bbecbb63"
*** Checking out PMKAlamofire at "3.2.3"
Segmentation fault: 11

Actual outcome
Crash with Segmentation Fault: 11

Expected outcome
No crashes

carthage_2019-04-05-221015_EclipseSSD-4.crash.zip

[mdiep]

Thread 4 Crashed:: Dispatch queue: com.apple.root.user-interactive-qos.overcommit
0   libswiftCore.dylib              0x0000000103e32b80 swift_isUniquelyReferenced_nonNull_native + 0
1   carthage                        0x000000010305ef35 closure #1 in cloneRepository(_:_:isBare:) + 181
2   carthage                        0x000000010306e511 partial apply for closure #1 in cloneRepository(_:_:isBare:) + 17
3   carthage                        0x00000001032d3d73 closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 387
4   carthage                        0x00000001032e799d partial apply for closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 61
5   carthage                        0x00000001032b8d17 Signal.Observer.send(_:) + 39
6   carthage                        0x00000001032cf898 Signal.Core.tryToCommitTermination() + 808
7   carthage                        0x00000001032cd90b Signal.Core.send(_:) + 715
8   carthage                        0x00000001032e6719 partial apply for Signal.Core.send(_:) + 9
9   carthage                        0x00000001032b8d17 Signal.Observer.send(_:) + 39
10  carthage                        0x00000001032ee143 closure #1 in closure #1 in TransformerCore.start(_:) + 67
11  carthage                        0x00000001032fde4a partial apply for closure #1 in closure #1 in TransformerCore.start(_:) + 42
12  carthage                        0x00000001032fce60 partial apply for closure #1 in closure #1 in TransformerCore.start(_:) + 16
13  carthage                        0x000000010329b9bd closure #1 in closure #1 in static Signal.Event.map<A>(_:) + 381
14  carthage                        0x00000001032a745d partial apply for closure #1 in closure #1 in static Signal.Event.attemptMap<A>(_:) + 45
15  carthage                        0x00000001032a7210 partial apply for closure #1 in closure #1 in static Signal.Event.map<A>(_:) + 16
16  carthage                        0x000000010329bcc3 closure #1 in closure #1 in static Signal.Event.mapError<A>(_:) + 451
17  carthage                        0x00000001032a6b6d partial apply for closure #1 in closure #1 in static Signal.Event.mapError<A>(_:) + 45
…

[mdiep]

It looks like we need a lock on FetchCache.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[taisukeh]

I also encountered the same problem. In my case, the cause of the problem was solved in the following PR.
https://github.com/Homebrew/homebrew-core/pull/38656

The removed mojave bottle was cause the crash.

[hlineholm]

Running rm -rf ~/Library/Caches/carthage and rm -rf ~/Library/Caches/org.carthage.CarthageKit seems to be a workaround.

In a Jenkinsfile:

sshagent(['CREDENTIAL_IDENTIFIER']) {
    lock('carthage') {
        script {
            try {
                sh 'carthage bootstrap --platform iOS --cache-builds'
            } catch (error) {
                echo error.toString()
                retry(1) {
                    sh 'rm -rf ~/Library/Caches/carthage'
                    sh 'rm -rf ~/Library/Caches/org.carthage.CarthageKit'
                    sh 'carthage bootstrap --platform iOS --cache-builds'
                }
            }
        }
    }
}

[TaborKelly]

Running rm -rf ~/Library/Caches/org.carthage.CarthageKit didn't fix my system.

[hlineholm]

@TaborKelly did you try rm -rf ~/Library/Caches/carthage? It should clear the Carthage database.

[TaborKelly]

Sorry, I forgot to reply to this. Yes, now it just doesn't work, but it also doesn't crash:

$ rm -rf ~/Library/Caches/carthage/  ~/Library/Caches/org.carthage.CarthageKit/
$ carthage bootstrap
*** No Cartfile.resolved found, updating dependencies
*** Cloning Proprietary1 
*** Cloning Proprietary2
*** Cloning Proprietary3
*** Cloning Proprietary4
*** Cloning Proprietary5
*** Cloning Proprietary6
*** Cloning Proprietary7
*** Cloning Proprietary8
*** Cloning Proprietary9
*** Cloning Proprietary10
*** Cloning Proprietary10
A shell task (/usr/bin/env git clone --bare --quiet [email protected]:WhereIWork/WhereIWork/Proprietary10.git /Users/tabor.kelly/Library/Caches/org.carthage.CarthageKit/dependencies/Proprietary10) failed with exit code 128:
fatal: cannot copy '/opt/local/share/git-core/templates/info/exclude' to '/Users/tabor.kelly/Library/Caches/org.carthage.CarthageKit/dependencies/Proprietary10/info/exclude': File exists

[d-date]

I also reproduced segmentation fault: 11

1. include repo with ssh e.g [email protected]/owner/reponame.git
2. carthage checkout (without --use-ssh)
3. carthage build --cache-builds --no-use-binaries --platform iOS

As workaround, use carthage checkout --use-ssh or carthage bootstrap --use-ssh ...

[igorkulman]

Same problem with 0.34.0, seems quite random

bash-3.2$ carthage bootstrap --platform iOS --cache-builds --no-use-binaries
*** Checking out PromiseKit at "6.8.4"
*** Checking out RxSwift at "4.5.0"
*** Checking out RxGesture at "1.2.1"
*** Checking out CleanroomLogger at "6.0.2"
*** Checking out NextGrowingTextView at "1.4.1"
Segmentation fault: 11
bash-3.2$ carthage bootstrap --platform iOS --cache-builds --no-use-binaries
now everything builds fine on second try

[bentoi]

We also get such SEGFAULTs sporadically but fairly often:

% make carthage
cd .. && carthage update --cache-builds
*** Fetching PromiseKit
*** Checking out PromiseKit at "6.11.0"
/bin/sh: line 1:  8459 Segmentation fault: 11  carthage update --cache-builds
...

The crash looks similar to the one from the previous comments:

Thread 2 Crashed:: Dispatch queue: /usr/bin/env git clone --quiet https://github.com/PromiseKit/Photos.git /Users/vagrant/workspace/ice/master/Carthage/Checkouts/
0   libswiftCore.dylib                  0x00007fff64fa7eb0 swift_isUniquelyReferenced_nonNull_native + 0
1   carthage                            0x000000010021eef8 specialized Dictionary._Variant.setValue(_:forKey:) + 40
2   carthage                            0x000000010021313c closure #1 in cloneRepository(_:_:isBare:) + 188
3   carthage                            0x000000010039387f closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 383
4   carthage                            0x00000001003a617d partial apply for closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 61
5   carthage                            0x0000000100379233 Signal.Observer.send(_:) + 35
6   carthage                            0x000000010038f5e8 Signal.Core.tryToCommitTermination() + 808
7   carthage                            0x000000010038d5c7 Signal.Core.send(_:) + 711
8   carthage                            0x0000000100379233 Signal.Observer.send(_:) + 35
9   carthage                            0x00000001003acf53 closure #1 in closure #1 in TransformerCore.start(_:) + 67
10  carthage                            0x00000001003bbfba partial apply for closure #1
...

[itsthejb]

Getting this here fairly often also...

[vytis]

I tried reproducing it by building Carthage from source but didn't manage to. I have a suspicion that it's related to few recent 0.34.0 and 0.33.0 formulas in Homebrew (starting from this commit). If I install Carthage from formula in a previous commit it works fine.

So the workaround is:

1. brew uninstall carthage
2. brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/d453dd956dbfc5b724b0fdaeb14f9480f461479d/Formula/carthage.rb

[OskarGroth]

My freshly set up App Center CI is unusable due to this error...

2019-11-13T15:13:29.8069780Z *** Checking out Defaults at "v3.1.1" 2019-11-13T15:13:29.8098630Z *** Cloning Defaults 2019-11-13T15:13:30.6745270Z /bin/sh: line 1: 909 Segmentation fault: 11 /usr/local/bin/carthage bootstrap --platform Mac 2019-11-13T15:13:30.7047640Z ##[error]/bin/sh failed with return code: 139 2019-11-13T15:13:30.7784980Z ##[error]/bin/sh failed with error: /bin/sh failed with return code: 139 2019-11-13T15:13:30.7985230Z ##[section]Finishing: Install Carthage frameworks

[zachwaugh]

We're seeing this frequently on CI (both CircleCi and GitHub Actions) for about the last two weeks or so. I don't think we changed/updated any dependencies in that time. Happening with version 0.34 and Xcode 11.2 (11.2.1) using carthage bootstrap --platform ios --cache-builds. It seems to be happening when checking out the dependencies, not during building. I have a script that tries to clear the caches if it fails and run again, and that doesn't seem to help at all.

[vytis]

@zachwaugh @OskarGroth We started getting the crash also quite recently on BuddyBuild. The reason was that Carthage got updated silently on the build machines. A similar thing might happen on other CI providers too I suppose. After doing the workaround mentioned above in our build scripts the problem went away.

[zachwaugh]

I noticed a common thread in the comments above is PromiseKit, which we also use. I just made a new empty directory with only this Cartfile:

github "mxcl/PromiseKit"

And got the segfault locally just from that

> carthage bootstrap --platform ios --cache-builds
*** No Cartfile.resolved found, updating dependencies
*** Cloning PromiseKit
*** Checking out PromiseKit at "6.12.0"
Segmentation fault: 11

Running it a second time worked, and I can't reproduce it reliably again locally. Changing the Cartfile to point to a specific release (6.10, 6.9, 6.8, etc) and running carthage update will sometimes, but not always segfault again.

PromiseKit has a ton of submodules - https://github.com/mxcl/PromiseKit/blob/master/.gitmodules, so I'm wondering that might be something Carthage isn't handling correctly in all cases.

[jdhealy]

Thanks for the additional fact-finding @zachwaugh — should be helpful in tracking this down.

[taisukeh]

https://github.com/Carthage/Carthage/blob/master/Source/CarthageKit/Git.swift#L37

This assignment should be executed mutually? The crash log shows the updateLastFetchTime called from two threads simultaneously.

Crashed Thread:        5  Dispatch queue: com.apple.root.user-interactive-qos.overcommit

Thread 5 Crashed:: Dispatch queue: com.apple.root.user-interactive-qos.overcommit
0   libswiftCore.dylib              0x00007fff665f5fdf _NativeDictionary.setValue(_:forKey:isUnique:) + 79
1   libswiftCore.dylib              0x00007fff665d4289 Dictionary._Variant.setValue(_:forKey:) + 281
2   libswiftCore.dylib              0x00007fff667c1220 specialized Dictionary.subscript.setter + 304
3   libswiftCore.dylib              0x00007fff665a0302 Dictionary.subscript.setter + 18
4   carthage                        0x000000010c38236d static FetchCache.updateLastFetchTime(forURL:) + 509 (Git.swift:37)
5   carthage                        0x000000010c383f78 closure #1 in cloneRepository(_:_:isBare:) + 56 (Git.swift:96)
6   carthage                        0x000000010c5cbe3f closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 1327 
....

Thread 6:: Dispatch queue: com.apple.root.user-interactive-qos.overcommit
0   libswiftCore.dylib              0x00007fff66836c93 swift_getGenericMetadata + 339
1   libswiftCore.dylib              0x00007fff667b212e specialized static _DictionaryStorage.resize(original:capacity:move:) + 254
2   libswiftCore.dylib              0x00007fff66696841 _NativeDictionary.resize(capacity:) + 129
3   libswiftCore.dylib              0x00007fff665f6178 _NativeDictionary.setValue(_:forKey:isUnique:) + 488
4   libswiftCore.dylib              0x00007fff665d4289 Dictionary._Variant.setValue(_:forKey:) + 281
5   libswiftCore.dylib              0x00007fff667c1220 specialized Dictionary.subscript.setter + 304
6   libswiftCore.dylib              0x00007fff665a0302 Dictionary.subscript.setter + 18
7   carthage                        0x000000010c38236d static FetchCache.updateLastFetchTime(forURL:) + 509 (Git.swift:37)
8   carthage                        0x000000010c383f78 closure #1 in cloneRepository(_:_:isBare:) + 56 (Git.swift:96)
9   carthage                        0x000000010c5cbe3f closure #1 in closure #1 in Signal.on(event:failed:completed:interrupted:terminated:disposed:value:) + 1327 (Signal.swift:957)
...

[itsthejb]

@zachwaugh When it crashes here (on 0.34.0), it's also on PromiseKit

[antranapp]

I also have PromiseKit as dependency, but it crashes randomly, but mostly when checking out and not necessary with PromiseKit.

[JuLink]

I think the issue for me appeared when I switch my terminal to zsh. I've updated to Catalina a while back and use carthage without issue and last week I read the terminal for the first time and it was telling me to change to zsh. I have nothing against bash nor zsh so I've changed it and since then it crash very often with this message : 28860 segmentation fault carthage update --platform iOS --cache-builds

[zachwaugh]

I've been running a build with the fix from this PR (https://github.com/Carthage/Carthage/pull/2908) since Friday, and so far no more segfaults.

[itsthejb]

Observation: we have just removed PromiseKit from our project, and this appears to work around the 0.34.0 crashes. Fix in https://github.com/Carthage/Carthage/pull/2908 or similar seems important to have regardless, though

[iby]

This has been following me for ages when updating specific dependencies. Adding --new-resolver option makes it work, though I'm no longer up-to-date on what it does – I thought the "new resolver" was merged into the master ages ago? Unless there's another new resolver these days.

[nalexn]

@JuLink pointed out the right direction - I switched to sh and the carthage update ran with no issues afterward:

> sh
> carthage update --no-use-binaries --platform iOS

[keefertaylor]

2908 seems to fix things, but it's not released in a version of Carthage (AFAICT 0.34 was cut before that PR was merged to master). How are folks consuming this? Build from source?

Is there a timeline for a new release of Carthage? 0.34 was released in Nov 2019, nearly 4 months ago.

[tmspzz]

Fixed in 0.35.0