Carthage: Adding `--build-for-distribution` option to enable module stability on consumer-side

[ikesyo]

https://github.com/ReactiveCocoa/ReactiveSwift/issues/779

[tmspzz]

@ikesyo I recall in the past once recommending doing this via .xcconfig files.

@jdhealy and I were chatting about this, in regards to what community folks seem to be expected and what package vendors might get surprised by … in the original issue there are several points to tackle:

1) ReactiveSwift doesn’t have BUILD_LIBRARIES_FOR_DISTRIBUTION=YES
1) OP is expecting ReactiveSwift to be module stable.
1) OP is using a different version of the compiler with regards to the one that was used to build binary.

Let’s exclude what Carthage 0.34.0 doesn’t claim to cover:
1) OP is expecting ReactiveSwift to be module stable
1) OP is using a different version of the compile with regards to the one that was used to build binary (__note that OP built from source__)

I’m including using different versions of the compiler as not an issue WRT 0.34.0 since it will reject anything not build with the same compiler by default.

For >= 0.35.0 the last point might be a Carthage issue. On one hand, however, .swiftinterface integration has numerous logged bugs that even if fixed in future Xcodes, will still be relevant for as long as the buggy Xcodes are around.

On the other hand, --build-for-distribution would be quite ergonomic if used knowing the consequences. To me a problem with a .swiftinterface is even harder to find than realizing there’s the wrong version of Xcode selected for CLI builds. Putting the easy-flag option for Carthage to go-past-the-library-authors’-potential-intentions in potentially two regards:

• bugginess of the Xcode-11-tied-together library-evolution/module-stability
  
  
  
  • inherent complications of the Xcode-11-tied-together library-evolution/module-stability
  
  

…we’d likely serve users better by providing better safeguarding/messaging.

:postbox: As a side note, this is also a potential problem with current master. Probably Carthage should warn that a binary doesn’t contain .swiftmodules, or if it does, that it was built with a different version of the compiler.

@jdhealy’s concern with this is that we’d be silently endorsing building for distribution with a simple flag like --build-for-distribution when in reality it’s still quite risky thing to do.

All the above seems to indicate Carthage users might (regardless of a --build-for-distribution) need expectation-setting from Carthage with a crafted message to guide their path on why library vendors might avoid BUILD_LIBRARIES_FOR_DISTRIBUTION=YES
current BUILD_LIBRARIES_FOR_DISTRIBUTION=YES dangers
and past Xcode versions’ lack of module stability.

[jdhealy]

📺 · if you've experienced issues or _a complete lack of issues_ with frameworks under BUILD_LIBRARIES_FOR_DISTRIBUTION=YES, we'd love to hear from you.

Hit the thumbs up emoji on this message if your experience was positive…

Or if you had issues under BUILD_LIBRARIES_FOR_DISTRIBUTION=YES hit the thumbs down emoji… and please consider leaving a message in this thread with details about the framework and the commit/tag 🙏

[andersio]

As a framework user, I have made a in-house prototype to build 40+ dependencies (including RAS/RAC) under BUILD_LIBRARIES_FOR_DISTRIBUTION=YES using a custom tool. Our targets and test suites build with them fine. However, I haven't pushed the prototype beyond this.

That is mainly because Xcode build system doesn't seem to pick up any XCFrameworks in FRAMEWORK_SEARCH_PATHS — only those being added via Xcode GUI are recognised. Since this constitutes a workflow change which I'd want to avoid, I've fallen back to a per-SDK folder layout akin to Carthage/Builds, and those are built without BUILD_LIBRARIES_FOR_DISTRIBUTION=YES.

---

As a framework author, I would expect not having to enable BUILD_LIBRARIES_FOR_DISTRIBUTION by default, even if we are to guarantee module stability. I would also expect existing compiler version detection in Carthage to continue to work as is, or in other words, continue to reject binaries built with different compilers except for Library Evolution frameworks.

Quoting the official Swift blogpost:

https://swift.org/blog/library-evolution/

Frameworks that are always built and distributed together, such as Swift Package Manager packages or binary frameworks that are internal to your app, should not be built with library evolution support.

(emphasis mine)

Building with the Library Evolution mode imposes a resilience boundary on the module, which incurs the extra runtime overhead.

CocoaPods appears to have chosen to mark transitive source dependencies of any XCFramework with BUILD_LIBRARIES_FOR_DISTRIBUTION=YES, so that they are built in Library Evolution mode only when required. Meanwhile, I cannot find any information on how SwiftPM deals with these situations.

Going back to the mentioned ReactiveSwift issue, this does look like a typical Carthage usage issue, which binaries are incompatible when the CLI tool uses a different toolchain from the IDE. I am not sure what we can do with this, other than highlighting these common issues in docs, or somehow detecting this & warning users at carthage build/boostrap time.

[tmspzz]

@thecivillain I think this is off-topic. Please keep the discussion on topic.

[moveitonover]

@tmspzz Sorry about that, I misread an earlier comment and thought it was related. I removed it to avoid confusion.

[1oo7]

So it should be viable to add "BUILD_LIBRARY_FOR_DISTRIBUTION=YES" to a Carthage workaround .xcconfig script in order to get forwards-compatibility? Would we also need to add "SKIP_INSTALL=NO"? Thanks

[jenhausu]

Please add this feature, now I have to turn BUILD_LIBRARIES_FOR_DISTRIBUTION to YES manual.