Cocoapods: Xcode10 (new build system) - if incremental build, Embed script doesn't run

[mkubista]

READ THIS FIRST (08/23/2019 update by @dnkoutso)

UPDATE 06/22/2020

Xcode 12 also claims to fix the issue for xcfilelists! (https://developer.apple.com/documentation/xcode-release-notes/xcode-12-beta-release-notes)

For CocoaPods > 1.8.0.beta.1 a fix was merged in #9125 by using USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES instead of ALLOW_RECURSIVE_SCRIPT_INPUTS that Xcode 11 Beta notes have mistakenly said.

_However_, #9125 still does NOT work if your project uses xcfilelists which CocoaPods utilizes automatically and as long as your project is Xcode 9.3 compatible.

An additional radar has been filed to Apple for xcfilelist to work with USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES.

---

• [x] I've read and understood the CONTRIBUTING guidelines and have done my best effort to follow.

Report

What did you do?

ℹ I made changes in some code files in local development pod

What did you expect to happen?

ℹ After build and run in xCode, the changes are incorporated into the resulting package.

What happened instead?

ℹ The old binary is in package. The [CP] Embed Pods Frameworks script doesnt run. Its because of build optimalisation in new xCode - when it detects no change in input or output files specified for script, the script is not started.
When we did enter the binary file from the content of the .framework folder into the input files list, everything is OK. Clean build is also OK.
So the solution would be to have no input and out files at all, or during pod install add also content files of frameworks into the input files list.

Strange thing - in the build log in Report navigator, the output of the [CP] Embed Pods Frameworks appeared even when it didn't actually run (I modified the script to echo current time, so I can see the same time stamps)

CocoaPods Environment

ℹ

   CocoaPods : 1.6.0.beta.1
        Ruby : ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
    RubyGems : 2.6.11
        Host : Mac OS X 10.14 (18A371a)
       Xcode : 10.0 (10L232m)
         Git : git version 2.17.1 (Apple Git-112)
Ruby lib dir : /usr/local/Cellar/ruby/2.4.1_1/lib
Repositories : artsy - https://github.com/Artsy/Specs.git @ 8c00e26ca1bd8ba163587f0eb592e9cd5a241d0e
               master - https://github.com/CocoaPods/Specs.git @ 058895fdadffd84d53c3ddc27a31591ea2f89ce7
               SSCPWrapper - https://tfs.kswr.cz/tfs/DefaultCollection/Jablotron/_git/MyCompany-iOS @ 518f383f1524b77aad7ddaa1d50eb72e4b824594

Project that demonstrates the issue

ℹ Unfortunately, project cannot be shown in public.

[Legoless]

I have the same issue. This causes me having to rebuild the entire project, which takes about 15 minutes. But I only changed a single line in code in one of the development pods.

[mkubista]

I have a possible temporary solution
I added this Run script phase before Embed Pods Framework
touch "${PODS_ROOT}/*SCRIPT_DIRECTORY*frameworks.sh"
The real dirctory can be found in XCODE log of the build.
It forces xcode to run the script everytime. Because inside is used rsync command, the time to do this every-time isn't as much long

[dnkoutso]

Even in the current build system the input/output files are used to speed up builds. If nothing has changed then there is no need to run these scripts.

Please include a sample app demonstrating this issue as I cannot reproduce locally.

[mkubista]

Here is the sample project https://github.com/mkubista/CocoapodsTest
The problem can be simulated by this steps:

1. Build and run the project - the viewcontroller with "Test 2" label should appear.
2. Change the code in cocoapods module in CocoapodsTest.Submodule/CocoapodsTestSubmodule/Utils.swift for example to return "Test 1" instead.
3. Build and run again - one would think the "Test 1" will appear but "Test 2" is here. Clean build will solve this.

Project is in git with all necessary files.
I hope the problem is not on my computer only (if so I have no clue what could cause this)

[djbe]

I have the same issue in a large project with around 6 development pods.

Whenever we modify anything in a development pod, we are forced to perform a full clean & build before the changes are propagated on device (or in simulator).

[dnkoutso]

Thanks for sample will take a look when time permits.

[dnkoutso]

Confirmed.

[dnkoutso]

Quick debugging shows its a new build system issue only and despite the file timestamp changing Xcode does not treat the resulting framework as "dirty" and therefore skips the script phase...

Will try to see if I can work around it but unfortunately this might need to be a radar instead.

[dnkoutso]

This is an unfortunate regression in Xcode 10 new build system. Switching to legacy system makes it work just fine.

I believe Apple is aware of this. Switching the input/output paths to point to the actual executable inside the framework makes the problem go away _but_ we may not want to do that. This means that the script phases will re-run correctly as long as the sources of the framework change but it won't catch resource changes or other things inside the .framework folder.

For now it seems that Xcode 10 build system will always require a clean build to ensure latest changes are present which sucks.

We could maybe add an option to not use input/output paths which would "fix" this but it is ultimately a workaround.

[mkubista]

I will try to report it in Apple Bug reporter. Will see, what they will say...

[maxkattner]

I created a radar (41126633) for that in June. They are aware, got a sample project and are working on fixing it. Unfortunately, I do not have any more information.
Feel free to created dupes as it helps them in prioritising!

[dnkoutso]

Thank you!!

[dmcgloin]

FYI: Tested on Xcode 10.1 beta (10O23u) and issue still occurs.

[dnkoutso]

I think the best thing for us to do is to provide an option not to use input/output paths. This will cause the script phase to run always but at least folks wont have to do clean builds.

[dnkoutso]

I opened up this PR (https://github.com/CocoaPods/CocoaPods/pull/8105) to provide an option to disable usage of input/output paths.

[brunophilipe]

Expanding on what @mkubista said, another solution is to add this as a run script phase before the "Embed Pods Frameworks" phase to automatically touch all framework shell scripts:

find "${SRCROOT}/Pods" -type f -name *frameworks.sh -exec bash -c "touch \"{}\"" \;

[dnkoutso]

This issue has been closed since we now merged #8105 which will ship with beta.2.

For anyone who is taking a look here, #8105 is not fixing the issue it just provides an option to disable input/output paths.

If you do use this option please note that you will get a performance hit on incremental builds since the script phases will constantly re-run but at least your build output should be correct.

There are a few workarounds present here but ultimately we hope Apple fixes the issue.

[dnkoutso]

@maxkattner do you have an open radar link to the issue you filed do you know the status of your issue if it has been updated or being worked on?

[brunophilipe]

As of now issue 41126633 is still open on bugreport.apple.com.

On 8 Oct, 2018, at 21:50, D. Koutsogiorgas [e-mail] wrote:

@maxkattner https://github.com/maxkattner do you have an open radar link to the issue you filed do you know the status of your issue if it has been updated or being worked on?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/CocoaPods/CocoaPods/issues/8073#issuecomment-427957909, or mute the thread https://github.com/notifications/unsubscribe-auth/ACCaQuhMT4HDH-pWTyLlABvLqTDQgNaSks5ui6x4gaJpZM4WXbfw.

[ralfebert]

Still happening with Xcode 10.1 beta 2.

Removed the in/output files in the 'Embed Pods Frameworks' manually as a workaround, this works for now.

Any hints how to install a cocoapod with the installation option from https://github.com/CocoaPods/CocoaPods/pull/8105 included? Do I need to checkout the repo and do 'rake install' or is there a smarter way? Is there a way to enable this globally as a default until the Xcode bug is fixed?

@maxkattner Do you have a link to the example project you attached? I'd like to create a dupe for the issue.

[amorde]

@ralfebert I recommend using Bundler to pin CocoaPods to the commit SHA or version that you need for your project

[maxkattner]

Hey @ralfebert, nice seeing you here! :D
I created a repository to check out the example project and dupe the radar: https://github.com/maxkattner/new-bs-bug-example

[ralfebert]

What strikes me as odd is that even with the in-/output files present for the "Embed Pods Frameworks" phase, I see the build phase being run in the build log and the files of the frameworks being rsynced to the app bundle (with the framework clearly not being up-to-date when running app the app). Any clues what in particular is causing this?

[jgavris]

@ralfebert I've found that very confusing, but it seems Xcode 'replays' the last output even in the incremental / no-op case.

[ralfebert]

Indeed, good to know! It helps to add a date command to the script to see if it's actually executed.

I had the suspicion that the issue might be that it doesn't pick up changes from files in a folder. I just did a quick experiment in a standalone project and if I use a folder as input file, the run script phase is not run when I change a file in that folder. But for this simple example, I get the same behaviour with the old build system (while in the Cocoapods project switching to the legacy build system clearly fixed the issue).

Any ideas what might make this work in the Cocoapods context but not in a simple project for the old build system?

I think it would help to get this fixed to show the regression in the most straightforward way. Unfortunately I could not find any documentation that specifies that this is supposed to work with a folder as input file.

[grigorye]

@ralfebert

What strikes me as odd is that even with the in-/output files present for the "Embed Pods Frameworks" phase, I see the build phase being run in the build log and the files of the frameworks being rsynced to the app bundle (with the framework clearly not being up-to-date when running app the app). Any clues what in particular is causing this?

Do you mean xcodebuild or the build log at Xcode report tab? If the latter, there's "All" and "Recent" buttons at the top left of build log window - I believe that "Recent" shows just what actually happened in the last build, no?

[dnkoutso]

Correct. Xcode will output in "All" the full script even though it actually didn't run it. The "Recent" tab will actually not include the script phase if it didn't run.

[ralfebert]

I documented the issue here:
https://www.ralfebert.de/ios/blog/cocoapods-clean-input-output-files/

I wondered, has it been considered to disable the input/output folders in the build phase by default until the Xcode bug is fixed?

[Fundebug]

@ralfebert It works, thanks.

[btelintelo]

Still seeing this issue in xcode 10.2

[Adrift001]

how to solve this question?

[amorde]

@Adrift001 there's an installation option to remove script paths which resolves this, but slows down incremental builds.

install! 'cocoapods', :disable_input_output_paths => true

Or, you can revert to the legacy build system and keep the input/output paths. See this radar for details

[Adrift001]

@amorde OK, thanks!

[dnkoutso]

This issue might be fixed in Xcode 11.

By default, Xcode’s new build system doesn’t detect changes in directories declared as inputs to shell script build phases. Enabling the build setting ALLOW_RECURSIVE_SCRIPT_INPUTS causes it to do so. However, if any of the files inside such a directory are generated by a task that depends on the output of the script phase, then a dependency cycle error is emitted and must be resolved by restructuring the target. (41126633)

Will require a cocoapods change to enable this setting.

Notes: https://developer.apple.com/documentation/xcode_release_notes/xcode_11_beta_release_notes

[dnkoutso]

It works only if the output file lists are removed. So only input paths.

[dnkoutso]

I take it back I dont think my initial test seems to work. oh well.

[nickgzzjr]

@dnkoutso Yup just tested it and doesn't work 😞

[forwyt]

Just click File->Project(WorkingSpack)Settings ->Build system (Legacy Build System) will make the problem ok !
it's work for me .

[igor-makarov]

How about Beta 4?

[dnkoutso]

Doesn't work on Xcode 11 Beta 4 either.

[maxkattner]

The official workaround is to set a user-defined build setting USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES to YES. This should already work with Xcode 11 Beta 1 and later versions. Sadly, the flag they announced in a previous changelog entry was wrong but this one now works just fine. Not sure if it there is the will to make this setting default or include it in the predefined build settings. Only time will tell. 🤷‍♂😄

[igor-makarov]

I tested this right now and it works.

I only have expletives.

[dnkoutso]

@maxkattner omg. I wish we would have known this earlier.........................! Was this a response from Apple to your radar?

[maxkattner]

One of their developers pointed me to the changelog and then noticed the mistake in its name provided in the document. I just checked and it is still wrongly named in the changelog.

[dnkoutso]

@maxkattner tried it, it works with normal input/output paths but doesn't work with xcfilelists.........

[maxkattner]

Did you already file a radar yourself? If not please do so and give them this additional info there. Thanks!

[dnkoutso]

I had one filed yes and updated it with that info. I think i might file a new one though as my existing one was marked as a duplicate.

[dnkoutso]

1.8.0.beta.2 was released that fixes this issue by using USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES but _only_ if xcfilelists are _not_ used.

[nickgzzjr]

@dnkoutso Is there a way to disable xcfilelists so that USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES works?

[nservidio]

Hey, sorry, I'm late to this discussion,

• I'm using Xcode Version 10.3 (10G8)
• pod Version 1.8.4
• find . -name '*.xcfilelist' yields no results in my project folder.

Adding this to my Podfile:

install! 'cocoapods', :disable_input_output_paths => true

Or setting my workspace build system to legacy does not allow me to pick up incremental changes in a "development" pod (pod 'foo', :path => '~/Developer/foo'). Cleaning doesn't seem to help either. I'm getting confused about all the different workarounds presented. Am I missing something?

Please let me know! Thanks!

[joaquin102]

This is going to solve all your issues. 100% confirmed!! https://stackoverflow.com/questions/50552752/how-to-rebuild-development-pod-changes

[acecilia]

@nickgzzjr To fix the issue you can disable xcfilelists by setting objectVersion to 48. You can do that with the following pre_install step to your Podfile (cocoapods only uses xcfilelists for objectVersion >= 50):

pre_install do |installer|
  # Workaround for https://github.com/CocoaPods/CocoaPods/issues/8073
  # For more context see https://github.com/CocoaPods/CocoaPods/issues/8073#issuecomment-628092129
  installer.analysis_result.targets.each do |target|
    project_path = target.user_project.path + "project.pbxproj"
    text = File.read(project_path)
    new_contents = text.gsub(/objectVersion = .*/, "objectVersion = 48;")
    File.open(project_path, "w") {|file| file.puts new_contents }
  end
end 

[dnkoutso]

Xcode 12 beta 1 claims to be fixing the issue with xcfilelists as well:

When using USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES with XCFileLists, the items within are now properly treated as recursive inputs. (54635196) (FB7109342)

Release notes https://developer.apple.com/documentation/xcode-release-notes/xcode-12-beta-release-notes

[dnkoutso]

CocoaPods has already been updated to use USE_RECURSIVE_SCRIPT_INPUTS_IN_SCRIPT_PHASES so we expect things to just start working.

[dnkoutso]

Confirmed Xcode 12 Beta 1 fixes this issue with the project given in comment https://github.com/CocoaPods/CocoaPods/issues/8073#issuecomment-418197075

Keeping this closed.

[teameh]

Haha you're fast 👍 😄