Readthedocs.org: Versions not being added on time

Created on 25 Oct 2017 · 21Comments · Source: readthedocs/readthedocs.org

I've had a few reports of versions not being added on webhook, and sometimes taking 1 to a few days to show up. For example, see #3185.

My guess:

I think that this is tied to our cloned repo on the build servers. For projects that aren't pushed to often, the cloned repo would be wiped if not touched in 1 day. Once the repo is wiped, a full clone gets the correct versions.

This will need verification

Bug more information replication

Source

agjohnson

All 21 comments

Any updates or advice on how to deal with this? I'm still trying to get a branch based on the last commit to successfully build to see if we can work out #3185, but that branch is just not being seen by RTD and it has been several weeks.

molpopgen on 7 Nov 2017

No update here yet. I'm not certain this is a bug yet. We'll update here when we look into this more.

agjohnson on 8 Nov 2017

Then is there a recommendation as to what to do? It seems that I'm unable to build either new or current branches on RTD at the moment. It seems hard to believe that this is not a real issue--all was working fine until recently.

molpopgen on 8 Nov 2017

Perhaps I should delete all webhooks and then reconnect after a day or two? What would the effect of that be?

molpopgen on 8 Nov 2017

Reconnecting webhooks has helped other cases, you might try that first.

agjohnson on 8 Nov 2017

I deleted the webhook and initiated a new one about 8 hours later. I deleted a new branch and re-pushed it, resulting in a "status 200" and the following output. Is any of this helpful?

I also see this in the "detail" section:

HTTP 405 Method Not Allowed
Allow: POST, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "detail": "Method \"GET\" not allowed."
}

Read the Docs
molpopgen▼
View Docs
Projects >
fwdpy11
Overview
Downloads
Search
Builds
Versions
Admin
Settings
Advanced Settings
Versions
Domains
Maintainers
Redirects
Translations
Subprojects
Integrations
Notifications
Advertising
Exchange
Response

Status: 200 Content-Type: text/html; charset=utf-8 Allow: POST, OPTIONS
{
  "build_triggered": false, 
  "project": "fwdpy11", 
  "versions": []
}
Request

Sslclientcipher: SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384, version=TLSv1.2, bits=256 Sslclientcertstatus: NoClientCert X-Github-Event: push X-Github-Delivery: 893b4518-c50c-11e7-86f2-863a9284d6c1 Connection: close Accept: */* User-Agent: GitHub-Hookshot/f1e70c3 X-Cluster-Client-Ip: 192.30.252.35 Sslsessionid: 7B1C9337D936886BA9558202916DAC85BFF5C858A73359FF5BC42195CD34ACF7 Host: readthedocs.org Content-Type: application/json
{
  "after": "0000000000000000000000000000000000000000", 
  "base_ref": null, 
  "before": "9acb311afb192eafd1251fef8f052277eaf3c495", 
  "commits": [], 
  "compare": "https://github.com/molpopgen/fwdpy11/compare/9acb311afb19...000000000000", 
  "created": false, 
  "deleted": true, 
  "forced": false, 
  "head_commit": null, 
  "pusher": {
    "email": "[email protected]", 
    "name": "molpopgen"
  }, 
  "ref": "refs/heads/last_working_RTD_commit_v2", 
  "repository": {
    "archive_url": "https://api.github.com/repos/molpopgen/fwdpy11/{archive_format}{/ref}", 
    "archived": false, 
    "assignees_url": "https://api.github.com/repos/molpopgen/fwdpy11/assignees{/user}", 
    "blobs_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/blobs{/sha}", 
    "branches_url": "https://api.github.com/repos/molpopgen/fwdpy11/branches{/branch}", 
    "clone_url": "https://github.com/molpopgen/fwdpy11.git", 
    "collaborators_url": "https://api.github.com/repos/molpopgen/fwdpy11/collaborators{/collaborator}", 
    "comments_url": "https://api.github.com/repos/molpopgen/fwdpy11/comments{/number}", 
    "commits_url": "https://api.github.com/repos/molpopgen/fwdpy11/commits{/sha}", 
    "compare_url": "https://api.github.com/repos/molpopgen/fwdpy11/compare/{base}...{head}", 
    "contents_url": "https://api.github.com/repos/molpopgen/fwdpy11/contents/{+path}", 
    "contributors_url": "https://api.github.com/repos/molpopgen/fwdpy11/contributors", 
    "created_at": 1490029317, 
    "default_branch": "master", 
    "deployments_url": "https://api.github.com/repos/molpopgen/fwdpy11/deployments", 
    "description": "Forward-time simulation in Python using fwdpp", 
    "downloads_url": "https://api.github.com/repos/molpopgen/fwdpy11/downloads", 
    "events_url": "https://api.github.com/repos/molpopgen/fwdpy11/events", 
    "fork": false, 
    "forks": 2, 
    "forks_count": 2, 
    "forks_url": "https://api.github.com/repos/molpopgen/fwdpy11/forks", 
    "full_name": "molpopgen/fwdpy11", 
    "git_commits_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/commits{/sha}", 
    "git_refs_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/refs{/sha}", 
    "git_tags_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/tags{/sha}", 
    "git_url": "git://github.com/molpopgen/fwdpy11.git", 
    "has_downloads": true, 
    "has_issues": true, 
    "has_pages": true, 
    "has_projects": true, 
    "has_wiki": true, 
    "homepage": null, 
    "hooks_url": "https://api.github.com/repos/molpopgen/fwdpy11/hooks", 
    "html_url": "https://github.com/molpopgen/fwdpy11", 
    "id": 85603555, 
    "issue_comment_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues/comments{/number}", 
    "issue_events_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues/events{/number}", 
    "issues_url": "https://api.github.com/repos/molpopgen/fwdpy11/issues{/number}", 
    "keys_url": "https://api.github.com/repos/molpopgen/fwdpy11/keys{/key_id}", 
    "labels_url": "https://api.github.com/repos/molpopgen/fwdpy11/labels{/name}", 
    "language": "C++", 
    "languages_url": "https://api.github.com/repos/molpopgen/fwdpy11/languages", 
    "master_branch": "master", 
    "merges_url": "https://api.github.com/repos/molpopgen/fwdpy11/merges", 
    "milestones_url": "https://api.github.com/repos/molpopgen/fwdpy11/milestones{/number}", 
    "mirror_url": null, 
    "name": "fwdpy11", 
    "notifications_url": "https://api.github.com/repos/molpopgen/fwdpy11/notifications{?since,all,participating}", 
    "open_issues": 4, 
    "open_issues_count": 4, 
    "owner": {
      "avatar_url": "https://avatars0.githubusercontent.com/u/6416159?v=4", 
      "email": "[email protected]", 
      "events_url": "https://api.github.com/users/molpopgen/events{/privacy}", 
      "followers_url": "https://api.github.com/users/molpopgen/followers", 
      "following_url": "https://api.github.com/users/molpopgen/following{/other_user}", 
      "gists_url": "https://api.github.com/users/molpopgen/gists{/gist_id}", 
      "gravatar_id": "", 
      "html_url": "https://github.com/molpopgen", 
      "id": 6416159, 
      "login": "molpopgen", 
      "name": "molpopgen", 
      "organizations_url": "https://api.github.com/users/molpopgen/orgs", 
      "received_events_url": "https://api.github.com/users/molpopgen/received_events", 
      "repos_url": "https://api.github.com/users/molpopgen/repos", 
      "site_admin": false, 
      "starred_url": "https://api.github.com/users/molpopgen/starred{/owner}{/repo}", 
      "subscriptions_url": "https://api.github.com/users/molpopgen/subscriptions", 
      "type": "User", 
      "url": "https://api.github.com/users/molpopgen"
    }, 
    "private": false, 
    "pulls_url": "https://api.github.com/repos/molpopgen/fwdpy11/pulls{/number}", 
    "pushed_at": 1510204328, 
    "releases_url": "https://api.github.com/repos/molpopgen/fwdpy11/releases{/id}", 
    "size": 680, 
    "ssh_url": "[email protected]:molpopgen/fwdpy11.git", 
    "stargazers": 3, 
    "stargazers_count": 3, 
    "stargazers_url": "https://api.github.com/repos/molpopgen/fwdpy11/stargazers", 
    "statuses_url": "https://api.github.com/repos/molpopgen/fwdpy11/statuses/{sha}", 
    "subscribers_url": "https://api.github.com/repos/molpopgen/fwdpy11/subscribers", 
    "subscription_url": "https://api.github.com/repos/molpopgen/fwdpy11/subscription", 
    "svn_url": "https://github.com/molpopgen/fwdpy11", 
    "tags_url": "https://api.github.com/repos/molpopgen/fwdpy11/tags", 
    "teams_url": "https://api.github.com/repos/molpopgen/fwdpy11/teams", 
    "trees_url": "https://api.github.com/repos/molpopgen/fwdpy11/git/trees{/sha}", 
    "updated_at": "2017-10-24T01:47:50Z", 
    "url": "https://github.com/molpopgen/fwdpy11", 
    "watchers": 3, 
    "watchers_count": 3
  }, 
  "sender": {
    "avatar_url": "https://avatars0.githubusercontent.com/u/6416159?v=4", 
    "events_url": "https://api.github.com/users/molpopgen/events{/privacy}", 
    "followers_url": "https://api.github.com/users/molpopgen/followers", 
    "following_url": "https://api.github.com/users/molpopgen/following{/other_user}", 
    "gists_url": "https://api.github.com/users/molpopgen/gists{/gist_id}", 
    "gravatar_id": "", 
    "html_url": "https://github.com/molpopgen", 
    "id": 6416159, 
    "login": "molpopgen", 
    "organizations_url": "https://api.github.com/users/molpopgen/orgs", 
    "received_events_url": "https://api.github.com/users/molpopgen/received_events", 
    "repos_url": "https://api.github.com/users/molpopgen/repos", 
    "site_admin": false, 
    "starred_url": "https://api.github.com/users/molpopgen/starred{/owner}{/repo}", 
    "subscriptions_url": "https://api.github.com/users/molpopgen/subscriptions", 
    "type": "User", 
    "url": "https://api.github.com/users/molpopgen"
  }
}
About Us

Team
Open Source Philosophy
Blog
Our Sponsors
Read the Docs

Getting Started Guide
Documentation
Contributing
Advertise with Us
Commercial Support
Read the Docs Gold
Donate
© Copyright 2010-2017, Read the Docs, Inc & contributors

Change Language

molpopgen on 9 Nov 2017

After some wrangling, the new branch finally did appear. Not sure what is different, but resetting the web hook a few times seems to have solved it.

molpopgen on 9 Nov 2017

I have the same problem with https://snakemake.readthedocs.io. Whenever I add a new tag marking a new stable release, it takes days to appear on readthedocs.

johanneskoester on 21 Dec 2017

@johanneskoester Deleting and regenerating the webhooks eventually helped for me, but I found it a little tricky to figure out exactly what to do. In fact, I'm not sure I could recreate what I did now! IIRC, you delete the hooks from the github side.

molpopgen on 21 Dec 2017

You can also delete the integration from our side if you've connected your project at some point in the recent past.

Closing this as I don't think this is actionable anymore and I haven't replicated it since. I think the solution is to reconnect webhook if you are having trouble with webhooks.

agjohnson on 31 Mar 2018

This issue is popping up again. After a long time of now pushes to branches that get built, I want to build a new branch, but it is not listed. having to delete hooks, etc., again.

molpopgen on 29 Jul 2018

@molpopgen can you please provide your rtd project?

stsewd on 29 Jul 2018

@stsewd https://readthedocs.org/projects/fwdpy11/

I don't really understand what the webhook is accomplishing. The RTD side lists branches that were deleted from GitHub over a year ago. New branches simply don't make it there.

I just manually deleted the webhooks from both the RTD and the GitHub side in the hopes that "checking if it is plugged in" will help.

molpopgen on 29 Jul 2018

Your latest build is from 4 months ago, rtd can't detect when a new branch/tag is added/deleted, you need to rebuild any version of your project to refresh the branches/tags

stsewd on 29 Jul 2018

Could that please be documented somewhere? That is not what the current docs imply.

molpopgen on 29 Jul 2018

👍1

@molpopgen yeah, that should be documented, I'll see where that can be placed. Let me know if rebuilding fix your problem with new/old branches

stsewd on 29 Jul 2018

@stsewd

Let me know if rebuilding fix your problem with new/old branches

Yes, that did fix it. The idea that you have to force a build of an existing branch on RTD in order to update the list of branches should be front and center in bold in the integrations part of the docs. This issue was closed without ever mentioning this, which could have save a lot of time. Rather, I probably fixed it by luck last time and then had to try to recreate that luck this time.

molpopgen on 30 Jul 2018

👍1

@molpopgen we are thinking in a way to solve this, listening to more GitHub events and trigger a resync, so no manual intervention is needed

stsewd on 30 Jul 2018

@stsewd would you mind creating an issue for this linking the github docs (and finding the docs for the other VCS that we support) + explaning what are the limitations in each case? Thanks! :)