Cocoapods: Pod install with Incremental installation does not update changed project file

[ignatovv]

• [x] I've read and understood the CONTRIBUTING guidelines and have done my best effort to follow.

Report

TLDR: incremental pod install does not check user changes of generated xcodeprjects

What did you do?

1. Create empty test project with single development pod
2. Turn on Incremental installation flag
3. Create git branch and checkout in it
4. Execute pod install
5. Add new file to development pod project via Xcode interface
6. Do _not_ execute pod install
7. Commit changes
8. Change branch back to master branch. No new file in this branch
9. Execute pod install
10. Build project

What did you expect to happen?

I expect CocoaPods to generate new development pod project without file added in another branch.

What happened instead?

File is missing from file system but still presents in devpod project file what causes compilation error.

CocoaPods Environment

CocoaPods env

### Stack ### Stack

   CocoaPods : 1.8.4
        Ruby : ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin16]
    RubyGems : 2.6.12
        Host : Mac OS X 10.14.6 (18G103)
       Xcode : 10.3 (10G8)
         Git : git version 2.20.1 (Apple Git-117)
Ruby lib dir : /Users/vyuignatov/.rvm/rubies/ruby-2.3.3/lib
Repositories : master - git - https://github.com/CocoaPods/Specs.git @ d0cb7e67ee8edcd0672f504df358c80bcc004c3e
               trunk - CDN - https://cdn.cocoapods.org/

### Installation Source

Executable Path: /Users/vyuignatov/.rvm/gems/ruby-2.3.3/bin/pod

### Plugins

claide-plugins          : 0.9.2
cocoapods-amimono       : 0.0.10.avito
cocoapods-avito-amimono : 0.0.14
cocoapods-deintegrate   : 1.0.4
cocoapods-dependencies  : 1.0.0.beta.1
cocoapods-plugins       : 1.0.0
cocoapods-search        : 1.0.0
cocoapods-stats         : 1.1.0
cocoapods-trunk         : 1.4.1
cocoapods-try           : 1.1.0
cocoapods_debug         : 0.1.0

### Podfile

install! 'cocoapods',
     :generate_multiple_pod_projects => true,
     :incremental_installation => true


target 'CacheDemo'
pod 'AFNetworking', '~> 1.0'
pod 'DevPod', :path => './DevPod'

Possible causes and solutions

Incremental pod install validate file system structure but not the xcodeproj file structure. If I add file to xcodeproj and then delete it from file system CocoaPods is blind on this changes.

• Execute pod install before changing the branch. This will generate new cache with new structure and after changing the branch it will be invalidated and new xcodeproj will be generated.
• Add some kind of generated project hash to .project_cache. This will invalidate cache if current project file is modified in any way.

Motivation

We have large distributed team of developers, with a lot of changes on a daily basis. Executing pod install before changing the branch is huge overhead and it is decreasing value of incremental installation dramatically. If you need spare hands to fix this issue I am happy to contribute to the community but I need some guidance on technical details.

Project that demonstrates the issue

You can see steps to reproduce in readme
https://github.com/ignatovv/CocoaPodsCacheDemo

[dnkoutso]

Yes I believe this is a known issue with development pods not being taken into consideration for incremental installation.

[ignatovv]

Can you provide any links or another issues related to this problem please, I want to track the progress. I did not find any clues on this by myself.

[dnkoutso]

Sorry there is no other issue except this one! I should have mentioned we've seen this issue internally not so much in public.

[beefon]

Are there any plans to fix it? With this issue in place the incremental pod install becomes much less convenient. It means you are not really allowed to mess with generated Xcode project files, but it is very convenient from developer perspective to add/remove files directly in Xcode instead of putting files on disk and then invoking pod install.

I assume CocoaPods could compute hash of generated Xcode project files, it wouldn't take too much time, but then it will be possible to match on disk state properly and decide should project be re-generated or not.

[dnkoutso]

No plans yet but it might happen for 1.9.x.

[dnkoutso]

The bug is that the project is not being regenerated but I believe there is _another_ issue in which the sandbox can go out of sync due to local development pod changes. I think it might be the same but I haven't verified.

Either way yes this one is definitely an issue that needs to be corrected.

[amorde]

Should we always mark development pods as dirty? It will slow down install but it seems like that's the _correct_ thing to do since things could change at any point without CocoaPods knowing what changed

[sebastianv1]

😬 Incremental would lose a lot of its power for projects that use mostly development pods if we always regenerate those targets.

The problem with taking a hash of the project is we've noticed that Xcode can sometimes write new objects to the project file even if nothing has changed during a save. This could cause over generation and likely break incremental installation with every Xcode update that includes new project elements, making it a really high maintenance feature. But maybe the performance hit is worth the correctness? Thoughts @dnkoutso ?

[dnkoutso]

I'd like to believe on the side that Xcode won't be doing too many crazy things and even if it does I suspect it would only affect the projects the user was messing around with.

I think we should try it with the project cache and see...I'd even er on the side of us trying it internally on our large project first before it gets upstreamed.

[dnkoutso]

I am currently removing it from 1.9.0 milestone as I dont think this will make it.

[beefon]

This adds a lot of inconvenience for us. We constantly see compilation errors after switching branches and performing pod install. Then we have to pod install --clean-install to make project buildable again. This adds a lot of pain to our developer experience.

I believe this issue is experienced by anyone who has local development pods and who has more than a single git branch.

[beefon]

Sorry there is no other issue except this one! I should have mentioned we've seen this issue internally not so much in public.

@dnkoutso If there is an open one, why people should file another bug report?

[sebastianv1]

@beefon One implementation detail of incremental installation is that it'll selectively regenerate pods whose projects are missing in the sandbox. So if you know the project DevelopmentPodA.xcodeproj is dirtied, you can run rm Pods/DevelopmentPodA.xcodeproj followed by pod install, and that will only regenerate that individual project instead of the entire workspace.

[beefon]

@sebastianv1 Sure, we've noticed that, but we have like 70+ dev pods and some 3rdparty pods, and it is hard to know which projects will change after rebasing your branch on top of develop. In some cases we can cut off a good minute from pod install thanks to incremental project generation, in others - we have to face compilation errors and do --clean-install.

[west0r]

It's really annoying bug. I can spend several hours to deal with compilation errors after branch switching. And other developers too. And I think we aren't alone with this issue.

It costs us many man-hours. Many human life hours. Why it's not priority?

[dnkoutso]

There is no sense of priority in dealing with issues. The project is strictly supported by contributors who spend their free time to try to tackle issues but there is no issue triage or planning process. I marked this issue for 1.10.0 for anyone who can help fix it.

[amorde]

@west0r if this is a significant issue for your team then maybe you could consider contributing improvements :)

@beefon if this is causing more friction for you than the decreased pod install time is worth, why not disable incremental installation?