Dependabot-core: Follow symbolic links for e.g. Gemfile.lock

[davidwessman]

Hello!

I have been experimenting a bit with running rails development in Docker and for one thing I tested I put the files required for building in a specific docker directory (to make the Docker build context smaller).

Therefore I put Gemfile and Gemfile.lock in a folder called docker.
I put symbolic links to both files in the app root directory.

However when Dependabot updated my ruby dependencies, it wrote the Gemfile.lock as a new file. Not at the place where the symbolic link led.
https://github.com/davidwessman/sync-hours/commit/b341cfd549cf426bd5b6684c95a7dc41b884e005

I can set it up differently to work with both docker and dependabot, so it is nothing crucial to my setup.

Thank you for a great service!

[greysteil]

Interesting! I've never seen that setup before.

I've just been investigating the way the GitHub API present symlinked files, and it makes it difficult for Dependabot to detect them. To do so we'd have to make a separate request to the GitHub API to check the file type in all cases, which isn't ideal.

It looks like you've switched your setup away from the above, so there's no urgent need for Dependabot to support it - is that right? If so I'm going to try to convince GitHub to tweak their API to make it clearer when a file is, in-fact, a symlink. Might take a while, but it will allow Dependabot to handle this case without needing to make additional requests.

[davidwessman]

😄 Might not be super useful, but I kind of assumed it would work.

Yes, I switched solution - so no rush! Thank you for responding.

[greysteil]

FYI, GitHub got back to me to say that the current API response they're serving us (which makes it difficult to detect whether a file is a symlink) is a bug. They're talking about fixing it internally - if/when that's done I'll get Dependabot working as you'd expect with symlinks.

🤞

[lokiwins]

Any updates on this? Wanted to implement this for our repo but it won't run Bundler due to symlink issue.

[rebelagentm]

Hi! 👋 The team is heads down for the next few months scaling Dependabot for all of GitHub :octocat:, so it may be a while before we are able to continue work on this.

[rebelagentm]

Update:

There is a fix on the GitHub side that is currently awaiting review.

[mlarraz]

Hi, any update on this?

I'm experiencing a similar issue, where a project pulls in an internal, unversioned gem located in a symlinked path, and Dependabot fails to follow the symlink to its version.rb

[rebelagentm]

@feelepxyz I know there was a fix being worked on the GitHub side. Do you know where it stands?

[mlarraz]

Hi, I made a simple example to reproduce the error I'm seeing: https://github.com/mlarraz/dependabot_symlink_test

Here's the issue dependabot created, with a link to the log: https://github.com/mlarraz/dependabot_symlink_test/issues/1

Here's the specific error:

404 https://api.github.com:443/repos/mlarraz/dependabot_symlink_test/contents/app/dependencies/foo/lib/foo/version.rb?ref=0ff10ea747602122669554c3089791d9b92a36af

Note that while https://api.github.com/repos/mlarraz/dependabot_symlink_test/contents/app/dependencies/foo/ returns the correct response, nothing beneath that does.

[mlarraz]

@rebelagentm @feelepxyz ping

[ashkulz]

This is blocking us, as the solution posted by @feelepxyz for dependabot/dependabot-core#2106 doesn't work with symlinks (which I'm having to use for Gemfile_next.lock).

[ashkulz]

@feelepxyz I'm willing to submit a PR, if the GitHub changes have been rolled out -- would appreciate pointers on to which API now results the metadata you need.