Renovate: git submodule support

[felixfbecker]

What would you like Renovate to be able to do?
Low priority, but just had the first use case ever for git submodules in one of my open source projects, would have loved to use Renovate.
https://github.com/felixfbecker/PowerShellXSD

Describe the solution you'd like
PRs to update git submodules

Describe alternatives you've considered
Currently using Dependabot

[k2snowman69]

I was legitimately looking for this today, great to see an issue on it! An addition to dependabot's solution that I think maybe useful is to allow syncing to the Releases feature of GitHub vs the commit SHA. I'm not sure how you would compare version numbers since releases don't require semver style formatting; a possible start is to attempt semver parse and if that fails go off date release was created.

Just a few thoughts on this feature! Thanks for such an awesome product!

[rarkins]

When implementing this, it should be done for gitFs-only. Right now I think we only look for "real" files and not symlinks or submodules. Source: https://github.com/renovatebot/renovate/blob/86586538724a78912503209388796d35dbda8505/lib/platform/git/storage.js#L155-L172

So either getFiles needs to be modified to return both files and submodules and differentiate between them with metadata, or perhaps we add a function to specifically just get submodules. It looks like git submodule status --recursive could be a command that does that.

Rather than try to shoehorn this concept into our existing getFiles/fileMatch concepts, maybe it's better to make it a special case and keep the code simpler.

Proposal:

• Add a getSubmoduleList() function to platform, supported by gitFs only
• Skip the usual getFiles/fileMatch/extract steps for this manager and instead do it directly

[rarkins]

Using @felixfbecker's above repo as an example:

$ git submodule--helper list
160000 8f3625d3bdf2bafd9fd8944813f7b5b543eba602 0   PowerShell-Docs
$ cat .gitmodules
[submodule "PowerShell-Docs"]
    path = PowerShell-Docs
    url = [email protected]:PowerShell/PowerShell-Docs
    branch = staging

The .gitmodules file shows that it tracks branch staging. The list command shows the exact commit it's currently tracking (which is 6 months behind). Surprisingly to me, git submodule update doesn't change anything.

Here's how it looks when ready to update:

$ git diff
diff --git a/PowerShell-Docs b/PowerShell-Docs
index 8f3625d..6bd91d0 160000
--- a/PowerShell-Docs
+++ b/PowerShell-Docs
@@ -1 +1 @@
-Subproject commit 8f3625d3bdf2bafd9fd8944813f7b5b543eba602
+Subproject commit 6bd91d0de4c75dfda157a28a6963838cfb70bb7c

[JamieMagee]

I was looking at this recently, and the best way I found to get the latest commit was with git submodule foreach git ls-remote | grep <branch>. So for the example repo it would be:

$ git submodule foreach git ls-remote | grep staging
From [email protected]:PowerShell/PowerShell-Docs
659ee449e6927b153dcc315e3b296eb8c73922c9        refs/heads/staging

This still requires parsing of .gitmodules, but it's already a more general solution than what dependabot is doing—their solution is limited to GitHub and GitLab.

[rarkins]

Now that we have 100% gitFs, this type of solution sounds quite possible

[renovate-release]

:tada: This issue has been resolved in version 19.71.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[slauger]

Anyone got this running with private repositories (via SSH)? I always get a dependency lookup error.

INFO: Failed to look up dependency test-repo (repository=misc/submodule-renovate-test,
packageFile=.gitmodules, dependency=test-repo)

This is my .gitmodules:

-bash$ cat .gitmodules 
[submodule "test-repo"]
    path = test-repo
    url = [email protected]:misc/test-repo.git
    branch = master

I've added an SSH agent to the renovate Container and the git submodule update --init part seem to work fine.

When using public repositories (e.g. on GitHub) everything just works as expected.

[JamieMagee]

@slauger Can you open a new issue for this issue?