Readthedocs.org: Git submodules intermittently failing to clone resulting in failed builds

Created on 8 Jun 2017 · 13Comments · Source: readthedocs/readthedocs.org

Details

Project URL: https://github.com/drud/ddev
Build URL (if applicable): Failed Build: https://readthedocs.org/projects/ddev/builds/5535145/
Read the Docs username (if applicable): ddev

Expected Result

The project consistently clones and builds successfully.

Actual Result

The project intermittently fails to build. The error on these failed builds is the following:

Failed to import project: Failed to get code from 'https://github.com/drud/ddev' (git clone): 1 git clone error output: Cloning into 'containers/docker.nginx-php-fpm'... Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Clone of '[email protected]:drud/docker.nginx-php-fpm.git' into submodule path 'containers/docker.nginx-php-fpm' failed

The submodule repositories for this project are all public, so there is no reason why a permission denied error should occur. If i manually rebuild the project in the UI, it will usually end up succeeding after 1-2 attempts. There does not seem to be any discernible pattern to the failures.

Support

Source

tannerjfco

All 13 comments

I have a similar issue but in my case the submodule is private and no attempt should be made to clone it.
In most cases the build succeeds, the failure rate is low.
What I am concerned with is future updates. If the behavior is made consistent, builds might fail every time.
So I would like to be able to specify which submodules should be cloned.

jeras on 8 Jun 2017

This is still occurring for us on this project, pretty much all the time. Every merge we make into master is almost guaranteed to result in a failed readthedocs build that we go in and manually rebuild in hopes of eventually succeeding. It would be helpful to get a response on this issue. Thanks

tannerjfco on 30 Jun 2017

@tannerjfco:

Same for me. If you happen to solve the issue, please, let me know how.

barseghyanartur on 19 Dec 2017

I think this issue is already solved, I was able to build your docs successfully. And your repository url must be https://github.com/drud/ddev.git, note the .git.

stsewd on 27 Dec 2017

👍1

@stsewd:

Yep. Thanks. I gave it another try yesterday and it worked.

barseghyanartur on 27 Dec 2017

Great!

RichardLitt on 27 Dec 2017

Trigger happy. @tannerjfco - can you confirm this works?

RichardLitt on 27 Dec 2017

😄1

@RichardLitt:

I had to set the Programming Language (in Settings) to Only Words (was Python) and then it worked. Without the change it keeps failing. Other repositories that I have (that do not have sub-modules) do work fine with Programming Language set to Python. It looks like the combination of the two causes problems.

barseghyanartur on 27 Dec 2017

👍1

@RichardLitt unfortunately this does not appear to be resolved. While the error message has changed, it looks to be the same problem, with the same behavior. I can attempt rebuilds and eventually get a success, but the issue will turn up again later.

Updated error message:
"There was a problem connecting to your repository, ensure that your repository URL is correct and your repository is public. Private repositories are not supported."

I have tested the repo URL both with '.git' at the end and without, and can't discern any difference between the two. The repo being cloned it public.

tannerjfco on 5 Jan 2018

Does not seems to be resolved for me neither. I've tried @barseghyanartur tip (changing Programming Language from python to Only words), bu with no success. I alos got a public repository.

Error is :

There was a problem connecting to your repository, ensure that your repository URL is correct and your repository is public. Private repositories are not supported.

When i try to build manually, it works about 1 time out of 3 or 4.

sylvainbeo on 15 Feb 2018

@sylvainbeo:

How I finally solved it (what works now) - I have a mirror repository on BitBucket (without git sub-module). The docs are built from there with no problems.

For BitBucket I use Mercurial. Thus, I commit twice (to both GitHub and BitBucket). I'm quite please with the way it works now.

barseghyanartur on 15 Feb 2018

@barseghyanartur Thanks for the feedback, we will try this.

sylvainbeo on 15 Feb 2018

The original problem seems to be solved. I'm closing this one in favor of #4043 that mention a possible error with the submodule URLs and relates to other issues on how to select the submodules to check out.

We have a temporal flag now that we can use to skip all the submodule checkout. In case anyone want to enable that, just ping me here and I will do it.

I'm closing this one.