Swiftlint: Swiftlint 0.39 library not loaded

[Oliver-Kirkland-Evoke]

New Issue Checklist

• [X] Updated SwiftLint to the latest version
• [X] I searched for existing GitHub issues

Describe the bug

swiftlint always crashes after updating to version 0.39 via Homebrew. We've tried this on 3 systems and seen the same issue.

We have tried installing "Swift 5 Runtime Support for Command Line Tools" as per the known issues but the systems are too new to allow this to be installed.

We have downgraded to 0.38.2 which is working fine.

Complete output when running SwiftLint, including the stack trace and command used

$ swiftlint lint
dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib
  Referenced from: /usr/local/bin/swiftlint
  Reason: image not found
Abort trap: 6

Environment

• SwiftLint version: 0.39
• macOS versions: 10.15.3 && 10.14.6

• Installation method used: Homebrew

• Are you using nested configurations?
  If so, paste their relative paths and respective contents.

• Which Xcode version are you using: 11.3
• Do you have a sample that shows the issue? Run `swiftlint"

[marcelofabri]

@Oliver-Kirkland-Evoke does it work if you try brew reinstall --build-from-source swiftlint?

[Doxxer]

Same issue, Installation method used: Cocoapods

[marcelofabri]

Same issue, Installation method used: Cocoapods

Can you post my details about your environment? I can image why using homebrew wouldn't work, but using CP should work fine 🤔

[PatrickGaissert]

We have the same problem. Integration is also via Cocoapods, Xcode 11.3.1.

[marcelofabri]

OK, so looks like the issue is that the binary is somehow hardcoding the path to the Xcode path from the machine that created the binary:

$ DYLD_PRINT_RPATHS=1 swiftlint help
RPATH failed expanding     @rpath/lib_InternalSwiftSyntaxParser.dylib to: /usr/lib/swift/lib_InternalSwiftSyntaxParser.dylib
RPATH failed expanding     @rpath/lib_InternalSwiftSyntaxParser.dylib to: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/lib_InternalSwiftSyntaxParser.dylib
dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib
  Referenced from: /usr/local/bin/swiftlint

I'm not sure how to fix that and won't have time to look into this until later, so I recommend using 0.38.2 for now.

[marcelofabri]

@nathawes do you have any ideas on how to fix this? Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

[RicardoKoch]

This update broke our CI build system.
Currently it's pulling always the last version.
I'm testing a fix by copying the latest Xcode and renaming it to: Xcode-11.3.1.app

[Oliver-Kirkland-Evoke]

FYI
We use autoupdate with HomeBrew to keep our development systems and CI system using the same version of SwiftLint and other tools. If using HomeBrew I recommend using:

brew uninstall swiftlint
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/203436f8567a3b6455264a1d896da5b1df1e50e6/Formula/swiftlint.rb
brew pin swiftlint

This will keep SwiftLint at version 0.38.2 until brew unpin swiftlint is run.

[rfree18]

My team is also experiencing this problem on Xcode 11.3.1. We're installing it with CocoaPods as well

[serges147]

My team as well experiencing this problem on Xcode 11.3.1. We're installing it via CocoaPods.

[nathawes]

@nathawes do you have any ideas on how to fix this? Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

@akyrtzi what's the right thing to do here?

[RicardoKoch]

I was able to temporarily fix this issue by renaming Xcode.app to Xcode-11.3.1.app (inside Applications).

[leifktaylor]

This broke my team's CI build.
We resolved the issue for the time being by locking our Swiftlint version to 0.38.2 in our podfile.

[bskendig]

brew reinstall --build-from-source swiftlint built a copy of 0.39 for me that works properly, so that's a workaround.

[akyrtzi]

Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

Unfortunately not possible. In order to get a tool that is portable (working on a different machine than it was built with) you need to copy lib_InternalSwiftSyntaxParser.dylib and have it alongside the tool. And the tool's rpaths should be able to find it relative to itself.

[marcelofabri]

Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

Unfortunately not possible. In order to get a tool that is portable (working on a different machine than it was built with) you need to copy lib_InternalSwiftSyntaxParser.dylib and have it alongside the tool. And the tool's rpaths should be able to find it relative to itself.

Thanks! I'll look into that later 🙌

[jpsim]

@marcelofabri SourceKitten grabs clang and sourcekit dylibs dynamically here: https://github.com/jpsim/SourceKitten/blob/master/Source/SourceKittenFramework/library_wrapper.swift

We should do the same with lib_InternalSwiftSyntaxParser.dylib.

[jpsim]

@marcelofabri can we revert the SwiftSyntax inclusions in 0.39.0 and cut a 0.39.1 release to unblock folks who want to be on the latest bleeding edge SwiftLint version?

[marcelofabri]

@marcelofabri can we revert the SwiftSyntax inclusions in 0.39.0 and cut a 0.39.1 release to unblock folks who want to be on the latest bleeding edge SwiftLint version?

I was planning to look into fixing the issue in the evening, but if you (or any other contributor) have the time before that, feel free to revert the changes and cut a release.

[jpsim]

ok, I'll revert & cut a new release.

IMO we should avoid copying lib_InternalSwiftSyntaxParser.dylib and vending it with SwiftLint. We've been able to avoid doing that for SourceKit & libclang and we should be able to avoid it for SwiftSyntax too.

[marcelofabri]

Thanks - and sorry for the mess!

[akyrtzi]

IMO we should avoid copying lib_InternalSwiftSyntaxParser.dylib and vending it with SwiftLint. We've been able to avoid doing that for SourceKit & libclang and we should be able to avoid it for SwiftSyntax too.

There is a significant difference here in that sourcekitd.framework and libclang.dylib cannot be separated from the toolchain they came from, they will stop functioning if you try to copy them out of the toolchain.

lib_InternalSwiftSyntaxParser.dylib is completely self-contained so it can be copied out. We are even considering having it be a SwiftPM binary dependency, in which case it may not be in the toolchain anymore (or at least building SwiftSyntax from source will not require having that library in the toolchain).

[jpsim]

Thanks for that useful insight @akyrtzi! If we re-vend the binary as part of SwiftLint distributions, this also impacts our software license, so we'll need to carefully consider the implications there.

[norio-nomura]

@marcelofabri SourceKitten grabs clang and sourcekit dylibs dynamically here:
https://github.com/jpsim/SourceKitten/blob/master/Source/SourceKittenFramework/library_wrapper.swift

We should do the same with lib_InternalSwiftSyntaxParser.dylib.

As far as I tested, lib_InternalSwiftSyntaxParser.dylib is not compatible between Xcode 11.3.1 and Xcode 11.4.
terminal.sh-session $ swift package reset $ swift build --configuration release … [18/18] Linking swiftlint $ install_name_tool -delete_rpath /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx .build/x86_64-apple-macosx/release/swiftlint $ DYLD_LIBRARY_PATH=/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx .build/x86_64-apple-macosx/release/swiftlint lint --enable-all-rules --no-cache --quiet >&- Never call this for file that sourcekitd fails.: file SwiftLintFile+Cache.swift, line 185 fish: 'DYLD_LIBRARY_PATH=/Applications…' terminated by signal SIGABRT (Abort)
So it needs to check if the version of lib_InternalSwiftSyntaxParser.dylib is compatible with swiftlint before dynamic loading.

[jpsim]

Yeah that's what @akyrtzi mentioned above in https://github.com/realm/SwiftLint/issues/3105#issuecomment-584850304

[marcelofabri]

I was able to dynamically load it, but executing the binary still fails because of dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib (as @rpath/lib_InternalSwiftSyntaxParser.dylib is still mentioned in otool -L swiftlint) 🤔

[norio-nomura]

It needs to mark lib_InternalSwiftSyntaxParser.dylib as weak import like:
terminal.sh-session swift build --configuration release -Xlinker -weak-l_InternalSwiftSyntaxParser

[kylebrowning]

Confirmed, broken on CI.

[jpsim]

This issue shouldn’t happen with 0.39.1, which removed SwiftSyntax integration until we can improve the packaging situation.

[fdiaz]

I discovered a similar issue in a library I'm writing. A workaround was to rename Xcode-xyz.app to Xcode.app. It seems there's something in SwiftSyntax that assumes that swift toolchain will be under /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift.

I'm still investigating what's going on, but thought I'd share in case it's useful to anybody.

[jpsim]

Closing as we resolved this in 0.39.1 and 0.39.2.