Cocoapods: Incorrect OTHER_LDFLAGS for xcframeworks containing static binaries (beginning 1.10.0 release)

[johnbolka]

• [X] I've read and understood the CONTRIBUTING guidelines and have done my best effort to follow.

Report

What did you do?

Project setup:
An iOS app that consumes a development pod that has two vendored_frameworks that are both xcframeworks (both using static binaries)

The xcframeworks both have the same naming convention, so i'll provide one for simplicity.

Whats wrong? Beginning with cocapods v1.10.0, the ./Pods/Target Support Files/TheDevelopmentPodTarget/TheDevelopmentPodTarget.{release|debug}.xcconfig has an incorrectly generated OTHER_LDFLAGS. They were correct in cocapods v1.10.0.rc.1

What is the correct (previous) outcome under v1.10.0.rc.1:
OTHER_LDFLAGS = $(inherited) -l"MYXCFramework-iOS-XX-1.35.0" -l"MYOtherXCFramework-iOS-XX-1.35.0" -l"c++" -framework "CoreBluetooth" -framework "UIKit"

What is the current (wrong) outcome under v1.10.0:
OTHER_LDFLAGS = $(inherited) -l"mYXCFramework-iOS-XX-1.35" -l"mYOtherXCFramework-iOS-XX-1.35" -l"c++" -framework "CoreBluetooth" -framework "UIKit"

Note that the name of the xcframework was incorrectly changed (case change and clipping of version number) MYXCFramework-iOS-XX-1.35.0 vs mYXCFramework-iOS-XX-1.35

As a result, trying to compile the app fails as the library was not linked (since the name was incorrect). Locally changing the name to the correct name of the xcframework fixes the issue and the app compiles and runs.

What did you expect to happen?

Correct Result (as was seen in v1.10.0.rc.1):
OTHER_LDFLAGS = $(inherited) -l"MYXCFramework-iOS-XX-1.35.0" -l"MYOtherXCFramework-iOS-XX-1.35.0" -l"c++" -framework "CoreBluetooth" -framework "UIKit"

What happened instead?

Incorrect Result (as seen in v1.10.0)
OTHER_LDFLAGS = $(inherited) -l"mYXCFramework-iOS-XX-1.35" -l"mYOtherXCFramework-iOS-XX-1.35" -l"c++" -framework "CoreBluetooth" -framework "UIKit"

CocoaPods Environment

Stack

   CocoaPods : 1.10.0
        Ruby : ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
    RubyGems : 3.0.3
        Host : Mac OS X 10.15.7 (19H2)
       Xcode : 12.1 (12A7403)
         Git : git version 2.24.3 (Apple Git-128)
Ruby lib dir : /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib
Repositories : master - git - https://github.com/CocoaPods/Specs.git @ d5ef74c789dcf2f7641574c16b73fa7b22f22770

Installation Source

Executable Path: /usr/local/bin/pod

Plugins

cocoapods-binary      : 0.4.4
cocoapods-deintegrate : 1.0.4
cocoapods-plugins     : 1.0.0
cocoapods-search      : 1.0.0
cocoapods-stats       : 1.1.0
cocoapods-trunk       : 1.5.0
cocoapods-try         : 1.2.0
slather               : 2.5.0

Project that demonstrates the issue

I'd love to make a sample project, but I cannot find any publicly available xcframeworks that use static libraries. If you know of a sample project let me know and I'll make a sample project.

[dnkoutso]

Not sure if its related to https://github.com/CocoaPods/CocoaPods/pull/10122 cc @johntmcintosh perhaps?

[johntmcintosh]

@dnkoutso I took a look for a few minutes this morning, and I think this is related to the changes that merged in #10072.

With my sample test-case I am seeing that in the released 1.10.0 I have the following in the xcconfig:

OTHER_LDFLAGS = $(inherited) -l"coconutLib"

After walking back through the commits after the release of 1.10.0.rc.1, I found that the output changed to that format in 655cdc408 (when #10072 merged). Before that PR, the output I'm seeing was:

OTHER_LDFLAGS = $(inherited) -l"CoconutLib" -l"coconutLib"

I'm not immediately sure why the output before that PR contained what looks like both the correct and incorrect formats. That could be an artifact of something in my test case being incorrect, but I'm fairly sure that the fix will be related to the changes from that PR in lib/cocoapods/target/build_settings.rb.

---

Some notes on my test steps to help me or anyone else remember this later...

I modified my example project (Vendored-XCFramework12-Example/CoconutLib) to generate its CoconutLib as a static library by replacing CoconutLib.xcodeproj's framework target with a new static library target. I then replaced the lines in the CoconutLib/build.sh script that generate the xcframework with updates that generate based on expecting the input to be static libraries:

args=""
for archive in "${archives[@]}"; do
    args="$args -framework ${archiveDir}/${archive}.xcarchive/Products/Library/Frameworks/CoconutLib.framework"

    # Append -debug-symbols argument for this archive's dSYM
    args="$args -debug-symbols ${pwd}/${archiveDir}/${archive}.xcarchive/dSYMs/CoconutLib.framework.dSYM"

    # Append -debug-symbols argument for this archive's BCSymbolMaps, if they exist (device builds only)
    bcsymbolMapDir="${archiveDir}/${archive}.xcarchive/BCSymbolMaps"
    if test -d "${bcsymbolMapDir}"; then
        for symbolMap in "${bcsymbolMapDir}"/*; do
            args="$args -debug-symbols ${pwd}/${symbolMap}"
        done
    fi
done

echo "xcodebuild -create-xcframework $args -output 'build/CoconutLib.xcframework'"
xcodebuild -create-xcframework $args -output 'build/CoconutLib.xcframework'

with the following (skipping anything to do with debug symbols for simplicity of this test):

args=""
for archive in "${archives[@]}"; do
    args="$args -library ${archiveDir}/${archive}.xcarchive/Products/usr/local/lib/libCoconutLib.a"
done

echo "xcodebuild -create-xcframework $args -output 'build/CoconutLib.xcframework'"
xcodebuild -create-xcframework $args -output 'build/CoconutLib.xcframework'

Running CoconutLib/build.sh should now generate an xcframework that's similar in structure to what @johnbolka is describing.

Re-run bundle exec ../../bin/pod install from the CocoaPods/examples/Vendored-XCFramework12-Example directory and inspect the output of CocoaPods/examples/Vendored-XCFramework12-Example/Pods/Target Support Files/CoconutLib/CoconutLib.debug.xcconfig.

[dnkoutso]

cc @Westacular

[dnkoutso]

Anyone here who can help with this?

[paulb777]

A reproducible example would be a good first step.

[Skoti]

@paulb777
A reproducible example is provided by @johntmcintosh in one of the comments above.
I followed his description and when building _VendoredXCFrameworkExample_ I get:
CocoaPods-master/examples/Vendored-XCFramework12-Example/VendoredXCFrameworkExample/ViewController.swift:2:8: No such module 'CoconutLib'
Mind that the new static library target has Defines Module set to Yes.

The xcframework file is named _CoconutLib.xcframework_ and the lib file is named _libCoconutLib.a_.
The generated xcconfig file CocoaPods/examples/Vendored-XCFramework12-Example/Pods/Target Support Files/CoconutLib/CoconutLib.debug.xcconfig is as follows:

CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = NO
CONFIGURATION_BUILD_DIR = ${PODS_CONFIGURATION_BUILD_DIR}/CoconutLib
FRAMEWORK_SEARCH_PATHS = $(inherited) "${PODS_ROOT}/../CoconutLib/build"
GCC_PREPROCESSOR_DEFINITIONS = $(inherited) COCOAPODS=1
HEADER_SEARCH_PATHS = $(inherited) "${PODS_XCFRAMEWORKS_BUILD_DIR}/CoconutLib/Headers"
LIBRARY_SEARCH_PATHS = $(inherited) "${PODS_XCFRAMEWORKS_BUILD_DIR}/CoconutLib"
OTHER_LDFLAGS = $(inherited) -l"coconutLib"
PODS_BUILD_DIR = ${BUILD_DIR}
PODS_CONFIGURATION_BUILD_DIR = ${PODS_BUILD_DIR}/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)
PODS_ROOT = ${SRCROOT}
PODS_TARGET_SRCROOT = ${PODS_ROOT}/../CoconutLib
PODS_XCFRAMEWORKS_BUILD_DIR = $(PODS_CONFIGURATION_BUILD_DIR)/XCFrameworkIntermediates
PRODUCT_BUNDLE_IDENTIFIER = org.cocoapods.${PRODUCT_NAME:rfc1034identifier}
SKIP_INSTALL = YES
USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES = YES

Mind that both xcframework and library file name are simple and do not contain any version specifiers like in the original description of this issue and it still does not compile. Seems like the issue is not only about naming.

[dnkoutso]

This is the only issue blocking 1.10.1 release currently.

[paulb777]

If no one else gets to it, I should have the bandwidth to investigate within the next week.

[dnkoutso]

thanks @paulb777 !

[paulb777]

@johntmcintosh Would you commit the reproducible example to a branch? I'm not immediately able to follow to reproduce and don't want to spin on making the repro case.

Also a question perhaps for @amorde: When should xcframeworks use FRAMEWORK_SEARCH_PATHS and when should they use OTHER_LD_FLAGS. The Firebase pods using static xcframeworks seem to work fine accessing the frameworks via FRAMEWORK_SEARCH_PATHS with "${PODS_XCFRAMEWORKS_BUILD_DIR}/FirebaseAnalytics" being generated there.

[johntmcintosh]

@paulb777 Sure thing! I should be able to spend a few minutes tomorrow morning (US central time) making sure I remember what I tried 😆 and then will push that up to a branch you can work with

[amorde]

xcframeworks can wrap static libraries and in those cases FRAMEWORK_SEARCH_PATHS will not suffice AFAIK. its been difficult to handle every permutation correctly.

Firebase is working correctly because you're wrapping static frameworks instead of static libraries.

I apologize for my absence here - have been trying to get over burn out recently and don't have the energy to troubleshoot xcframework issues in the near term

[paulb777]

@johntmcintosh Thanks!

@amorde Thanks for the clarification. I didn't realize that xcframeworks supported static libraries. Good luck with the burn out! :)

[johntmcintosh]

Hi @paulb777, I've duplicated my original xcframework example directory, so you can work on this one in a standalone sandbox. It's worth noting I probably haven't made enough changes to the example for it to be "production-ready" to check into the main repo, but it should be a good starting point for your investigation.

Vendored-XCFramework12-Static-Example.zip

The example has an already-built CoconutLib as an xcframework wrapping a static library.

Try the following:

1. Checkout commit 655cdc408, which is where #10072 was merged.
2. Unzip the attached example into the examples directory.
3. cd into this new example directory:
   cd examples/Vendored-XCFramework12-Static-Example
4. Run the script I setup there which will re-install pods and output the OTHER_LDFLAGS value that seems suspicious:
   ./issue10165.sh
   For me, this outputs:
   OTHER_LDFLAGS = $(inherited) -l"coconutLib"
5. Checkout commit fe55affc1, which is one commit before #10072 was merged, and run through these same steps. When the script runs in this case, the output for me is:
   OTHER_LDFLAGS = $(inherited) -l"CoconutLib" -l"coconutLib"

Based on that, it looks to me like before that PR the OTHER_LDFLAGS line contained both the correct and incorrect values (and the incorrect was probably just failing silently for this type of use-case since the correct was also there), and after the PR, it only contains the incorrect value.

It's also worth noting that I haven't done a validation of anything else regarding compiling the Examples.workspace for this duplicated example, so it's possible that it may or may not also compile for other reasons if you try to test compilation there. To this point, my focus was just on validating the original report's comments about the OTHER_LDFLAGS value being changed to an unexpected value.

Hope this helps!

[Westacular]

Ah, I think I've figured this out:

In my PR, I updated build_settings.rb to create the linker flags using:

                xcframework_libraries = vendored_xcframeworks.
                                    select { |xcf| xcf.build_type.static_library? }.
                                    flat_map { |xcf| linker_names_from_libraries([xcf.slices.first.binary_path]) }.
                                    uniq

where linker_names_from_libraries trims the path and extension to get just the file name, and (if present) trims "lib" from the start of it.

However, binary_path in xcframework_slice.rb is already doing its own mangling of the path:

        def binary_path
          path + name
        end

where name for libraries is

            when :library
              result = File.basename(path, '.a').gsub(/^lib/, '')
              result[0] = result.downcase[0]
              result

So binary_path is really only meant for use on a .framework

Recommendations:

1) The quickest, simplest fix is to use xcf.slices.first.path instead of xcf.slices.first.binary_path at build_settings.rb:772, which will avoid the extra call to trim the file extension and the downcase of the first character.

2) We should probably also modify binary_path to return just path in the case of a library to avoid this footgun in the future

3) Is Slice.name used for library XCFramework slices anywhere else, and is the downcase actually needed for there? (APFS isn't case sensitive by default and I don't know how picky ld is in its library name matching when run on a case sensitive filesystem.)

[Skoti]

@johntmcintosh your CoconutLib target for static library is missing Defines Module build setting which should be set to Yes, otherwise xcodebuild will only generate .a file without any .swiftmodule files and consequently -create-xcframework action won't copy them.
And without swiftmodule files Xcode will fail a build when trying to import this library in a swift file i.e. in _Vendored-XCFramework12-Example/ViewController.swift_.

@Westacular your PR changes OTHER_LDFLAGS in config files: CoconutLib.release.xcconfig and CoconutLib.debug.xcconfig in _CocoaPods/examples/Vendored-XCFramework12-Example/Pods/Target Support Files/CoconutLib_
from $(inherited) -l"coconutLib" to $(inherited) -l"CoconutLib"
but it does not seem to have any effect, Xcode still reports "No such module CoconutLib" (even with Defines Module
enabled).

I've also noticed that:

• when you manually drag&drop an xcframework (with static lib that has defines module enabled) into "Frameworks, Libraries, and Embedded Content" then it simply works so it must be some configuration issue
• the build.sh script uses an archive action of xcodebuild which creates an .xcarchive file in the specified -archivePath but for some reason this archive file only contains .a static lib file without any swiftmodule files 😢 , we should copy them from the derived data folder and we could just use the build action (not sure why to archive?)

[dnkoutso]

Fixed and will ship with 1.10.1.