Carthage: `carthage update` fails if a submodule listed in .gitmodules doesn't actually exist

[robb]

I got this on the swift-development branch in the ReactiveCocoa repo, but I've seen it in Carthage, too.

$ carthage update
*** Fetching LlamaKit
*** Fetching Quick
*** Fetching Nimble
*** Fetching xcconfigs
Parse error: expected submodule commit SHA in ls-tree output:

Unfortunately, it's hard to reproduce. After a another invocation of carthage update, everything went as usual. Maybe have a --verbose mode could help tracking this down?

[joshaber]

I'm seeing this too.

[jspahrsummers]

I'm fairly convinced that this is the same as #139, so I'm gonna close this and track all info in that issue.

[justinmakaila]

+1

[justinmakaila]

I'm running into this on the newest version:

# Cartfile

github "Present-Inc/HLSKit" >= 1.1.6
github "Present-Inc/PresentAPIClient"
github "Present-Inc/PresentPrePermissions"
github "Present-Inc/Swifter"

github "Haneke/HanekeSwift"

github "ReactiveCocoa/ReactiveCocoa" == 2.4.2

github "justinmakaila/emitter-kit" >= 3.1

github "justinmakaila/facebook-ios-sdk" >= 3.21.3

Output:

$ carthage update --use-submodules
*** Fetching HLSKit
*** Fetching PresentAPIClient
*** Fetching PresentPrePermissions
*** Fetching Swifter
*** Fetching HanekeSwift
*** Fetching ReactiveCocoa
*** Fetching emitter-kit
*** Fetching facebook-ios-sdk
*** Fetching Alamofire
*** Fetching Prelude
*** Fetching SwiftyJSON
*** Fetching KVOController
*** Fetching Swell
*** Fetching xcconfigs
*** Fetching Quick
*** Fetching Nimble
*** Fetching Bolts-iOS
*** Fetching Swell
Parse error: expected submodule commit SHA in ls-tree output: 

[jspahrsummers]

@justinmakaila Does the repository in which you're running that command have any changes (as reported by git status)?

[justinmakaila]

Yes

[justinmakaila]

Committing and running carthage update worked

[jspahrsummers]

I'll reopen this. I think this occurs when a submodule has been added at a given path but not committed (because ls-tree operates on the tree of HEAD).

[akisute]

Struck on this issue without any workarounds. Looks like your guess is right because this only happens when I carthage update in my old repository that has added some submodules before but never happens on fresh git repositories.

# BAD
cd MyRepos
carthage update #fail

# GOOD
mkdir NewRepos
cp MyRepos/Cartfile NewRepos
cd NewRepos
git commit -am "initial import"
carthage update #success

[jspahrsummers]

@akisute Can you share the contents of your repository's .gitmodules file, if it has one?

[akisute]

Sorry I deleted .gitmodules when I was trying to fix this problem, but I can share you the repository I had the problem on: https://github.com/akisute/YourWalker

Here's a story: I have 2 PCs and I had a problem on the PC#1. During straggle of solution I found that I have unused .gitmodules in my repository, which submodules were located at old Carthage.build directory before migration. Anyway I deleted .gitmodules and tried again but didn't make it.

Now I'm on my PC#2, pulled the repository, tried carthage update, and there are no problems at all!

[s-aska]

My mistake. (version 0.5.0)

# main pc
echo "Carthage" >> .gitignore
carthage update --use-submodules
git commit ...
git push ...

# sub pc
$ git pull
$ carthage update
*** Fetching Foo
Parse error: expected submodule commit SHA in ls-tree output:

Was restored doing this ;)

$ git config -f .gitmodules --remove-section submodule.Carthage/Checkouts/Foo
$ git config -f .gitmodules --remove-section submodule.Carthage/Checkouts/Bar
$ git add .gitmodules
$ git commit -m 'reset carthage submodules'
[swift 7d5724d] reset carthage submodules
 1 file changed, 6 deletions(-)
$ carthage update

[aschuch]

Seeing this too, maybe you are able to reproduce the bug with the following example:

Trying to see if Moya works with Carthage.

• I have committed and pushed everything
• The project does not use .gitsubmodules

# Cartfile
github "AshFurrow/Moya"

(master|✔)➜ carthage update              
*** Fetching Moya
*** Checking out Moya at "0.6"
Parse error: expected submodule commit SHA in ls-tree output: 

[jspahrsummers]

@aschuch Do you have any submodules listed in .git/config?

[aschuch]

@jspahrsummers No, nothing. Does it work for you?

[jspahrsummers]

@aschuch Turns out that's an issue with Moya. See https://github.com/ashfurrow/Moya/pull/94.

---

Based on the above, it seems like carthage update will bail if a submodule listed in .gitmodules doesn't actually exist in the repository. We should probably silently ignore that case.

[aschuch]

@jspahrsummers Awesome, thanks. Maybe carthage update should display a warning, so people can fix their .gitmodules or open a pull request (like you did) if necessary?

[jspahrsummers]

I don't think it's appropriate for Carthage to warn about this when we can simply ignore it.

[pyrtsa]

@jspahrsummers Took me a while to figure out why it happened when this happened to us today. Turned out your diagnosis, buried deep in this GH issue, was right for us too:

I think this occurs when a submodule has been added at a given path but not committed (because ls-tree operates on the tree of HEAD).

If nothing else, maybe the error message could be improved to suggest this?

[jspahrsummers]

I agree that a better error message would be great. Want to submit one? :wink:

[toadums]

:+1: @pyrtsa. I just commited my branch and re-ran carthage update and it worked

[justinmakaila]

I'm experiencing this issue again on 0.7.4, only this time committing my work doesn't resolve it. Any suggestions?

[jspahrsummers]

@justinmakaila This isn't necessarily related to committing your work. This error will occur if .gitmodules on your repository's HEAD refers to a submodule path that doesn't exist.

[justinmakaila]

@jspahrsummers I cleaned my git config, .gitmodules and the .git/modules/ directory, and I'm still encountering this.

[jarrodrobins]

I am also encountering this issue in 0.7.5. I have one submodule to a private project which DOES exist at the given path, all changes are committed, yet, I still get the ol'

Parse error: expected submodule commit SHA in ls-tree output:

If I delete the submodule, removing it from .gitmodules and completely nuking it within the modules directory and out of my source directory, running carthage update works just fine.

As a horrible workaround I've taken my Cartfile, dumped it in a new directory on my desktop and run carthage update and then copy the output back into my repo.

[jarrodrobins]

I should also point out that I'll get the error even if my Cartfile is completely empty. If it helps - I DID have Moya in my Cartfile (it was mentioned before as potentially causing problems) but the same issue occurs even if I nuke it. Unless some remnant of it is sticking around when it shouldn't. It's certainly not in my .gitmodules.

[chrislconover]

In my case, I had to delete existing carthage entries from my .gitmodules file. I am using carthage 0.8.0.

[jarrodrobins]

After much bashing my head on the keyboard, I finally fixed (or rather, worked around my issue). I'm using 0.8.0.

This morning I deleted ALL entries from my .gitmodules file. I deleted all submodule directories, hell, I basically nuked my entire project. Nothing except the .git folder (with an EMPTY .gitmodules and NO .git/modules directory) and an EMPTY project directory (excluding the Carthage files). This damn issue still occurred.

As a final, grasping at straws last resort I MOVED my Carthage files up one directory. Previously, my .git repo looked something like this -

/
    .git
    MyProjectDirectory/
        Carthage
        Cartfile
        Cartfile.private
        Carthage/

I just moved all the Carthage directories up one level to put them in the root directory. Lo and behold, I can actually build my dependencies using Carthage again!

I suspect this is just a hacky fluke and the underlying issue is still there but hacky fluke or not, at least it's working again...

Can I suggest that the error message that Carthage spits out be improved? I'd be fascinated to know exactly what it's been whinging about this entire time.

[auibrian]

@jarrodrobins Check your .git/config file also for submodules that should not be in there? That seemed to work for me.

[toshi0383]

deleting .git/ worked for me, but nothing else..

[iliaskarim]

I just ran into this, and to the reports in this thread, worked around the issue by--

• removing stale submodules entries (deleted submodules) from BOTH .gitmodules AND .git/config
• moving Cartfile to the root of the github repo (which seems to be the recommended procedure, anyways, just wanted to try to reproduce @jarrodrobins's case)

after doing those two things, I was able to run both update and bootstrap

[skyline75489]

I ran into these, too. Here's my workaround:

• Remove .gitmodules
• Commit the changes
• Push it

Then carthage update finally works.

[mpurland]

I ran into the same issue as @jarrodrobins.

I tried many solutions including syncing .gitmodules, .git/config, and .git/modules and ensuring all of the git submodules were synced and up-to-date. I had a similar directory structure where the Cartfile was inside a subdirectory as it turns out.

After moving the Cartfile to the base directory of the git project carthage update works.

[chbeer]

Moving the Cartfile to the git base directory worked for me (i have the iOS app in a subfolder and tried to have the Cartfile in there)

Thanks, @mpurland

[tcurdt]

@skyline75489 got it right

I had to commit the delete of the .gitmodules file - now it works.

[DanielAsher]

this happened to me after removing an entry from the Cartfile

@tcurdt
deleting .gitmodules, commiting the delete and rerunning carthage update worked for me too.
thanks!

[robrix]

Identical issue with https://github.com/antitypical/Stream/commit/fe6f1b69296bd2cfff734f7dda786fc6743b5665:

rob@Resonance ~/D/G/s/p/E/Stream /(HEAD detached at fe6f1b6)> carthage update --verbose --no-build --use-submodules
*** Fetching Memo
*** Fetching Either
*** Fetching Prelude
Parse error: expected submodule commit SHA in ls-tree output: 

Identical workaround:

1. .gitmodules still contained a reference to robrix/Box (:interrobang:)
2. Manually removed the entry.
3. Committed this removal.
4. carthage update --no-build --use-submodules: success!

I’m not entirely sure where the ls-files invocation was even coming from; apparently we don’t pass the --verbose flag along to git invocations :confused: Looks like the error itself came from carthage, however.

Running git ls-files and git ls-files --ignored --exclude-standard manually didn’t reproduce the error.

[mdiep]

A reproducible case of this was shared in https://github.com/Carthage/Carthage/issues/847#issuecomment-152742111.

[robrix]

A reproducible case of this was shared in #847 (comment).

_Another_ reproducible case :wink:

[robrix]

So I think we’re doing git ls-tree -z HEAD Carthage/Checkouts/DependencyWhichNoLongerExists & it’s returning an empty string. Should we print out a warning there or just produce an empty signal producer?

[DanielAsher]

5 days ago I commented that removing .gitmodules and committing worked to fix this issue.

Sadly my repo seems to have been corrupted by this step. I created a new repo and transferred all files and ran carthage update --use-submodules successfully.

BEWARE!

[robrix]

Yeah, probably a bad idea to remove the .gitmodules file. Removing the faulty entry from it should be fine tho.

[robrix]

(And for future reference, removing submodules with git rm path/to/submodule should update .gitmodules correctly.)

[auibrian]

Hey @robrix, in my Sept 2nd message I was probably way lighter on details than was useful. I mentioned looking for submodules in .git/config that no longer were checked out. I figured this out by aliasing git to a shell script that printed out all the arguments and discovering the issue was with the ls-tree line. I tried to trace through the Carthage source to figure out why that particular command was needed but I didn't make a lot of headway and I was pressed for time to get a solution for building. Maybe going forward the solution is to ignore things in the .gitmodules or .git/config that aren't referenced in the Cartfile?

[robrix]

@auibrian: Yeah, that’s probably a good idea :+1:

[maximveksler]

What is the suggested work around? I've deleted my Carthage folder because I suspected that it was building projects I've removed from my Cartfile.private. Now I can't update the project at all carthage update gives Parse error: expected submodule commit SHA in output of task (ls-tree -z HEAD Carthage/Checkouts/CleanroomLogger) but encountered: any ideas about how I can fix this ?

[mdiep]

Try running git submodule update --init --recursive

[maximveksler]

Thanks @mdiep didn't work in my case #1146

[samuelpeter]

I have the following directory structure :

|-.git/
|-DemoAppProject/
    |-Cartfile
|-SDKProject/
    |-Submodule1/
    |-Submodule2/

Running carthage update from DemoAppProject results in :

Parse error: expected submodule commit SHA in output of task (ls-tree -z HEAD SDKProject/Submodule1) but encountered:

and indeed, running git ls-tree -z HEAD SDKProject/Submodule1 from DemoAppProject returns norhing. It does return something if run from the root directory. Running carthage update --project-directory DemoAppProject from the root directory does not solve the problem.

I ended up moving the Cartfile to the root directory just for the purpose of downloading/building dependencies, then moving everything back to DemoAppProject.

Seems to me that Carthage could ignore the empty output from git ls-tree altogether, since these submodules are not related to the project that uses Carthage...

[mdiep]

Ah, Carthage only supports Cartfiles in the root project directory.

[samuelpeter]

OK, thanks for the info @mdiep

[bhargavg]

1125 & #1150 should fix this.

Cartfile need not be in root directory anymore.

[samuelpeter]

Great thanks !

[ranchcoderz]

I fixed this by doing:

cd /git/repo/root/directory
git config -f .gitmodules --remove-section submodule.Alamofire
git add .
git commit -m"Fixing Alamofire"
git push

cd ./location/of/Cartfile
carthage update

[ranchcoderz]

I fixed this issue by leaving "shell" empty and putting "/usr/local/bin/carthage copy-frameworks" in the textarea

[ranchcoderz]

Finally got it to work as follows:

[hshah2811]

@skyline75489 .... Your solution worked for me.

I ran into these, too. Here's my workaround:

Remove .gitmodules
Commit the changes
Push it
Then carthage update finally works.