Renovate: Hosted Renovate Bot does not update any branches, because it receives 422, when setting "stability-days" status

[reckter]

What Renovate type, platform and version are you using?

Hosted Renovate on app.renovatebot.com + Github connection.

Describe the bug

It won't create PR's, or update them, auto merge them most of the time, because it encounters a 422, while trying to set the stability day status.

Relevant debug logs

}
DEBUG: Updating renovate/stability-days status check state to green(branch="renovate/definitelytyped")
DEBUG: Setting branch status(branch="renovate/definitelytyped")
{
  "context": "renovate/stability-days",
  "state": "green"
}
DEBUG: 422 Error thrown from GitHub(branch="renovate/definitelytyped")
{
  "err": {
    "name": "HTTPError",
    "timings": {
      "start": 1599206906787,
      "socket": 1599206906787,
      "lookup": 1599206906787,
      "connect": 1599206906858,
      "secureConnect": 1599206906932,
      "upload": 1599206906933,
      "response": 1599206907267,
      "end": 1599206907268,
      "phases": {
        "wait": 0,
        "dns": 0,
        "tcp": 71,
        "tls": 74,
        "request": 1,
        "firstByte": 334,
        "download": 1,
        "total": 481
      }
    },
    "body": {
      "message": "Validation Failed",
      "errors": [
        {
          "resource": "Status",
          "code": "custom",
          "field": "sha",
          "message": "sha must be a 40 character SHA1"
        }
      ],
      "documentation_url": "https://docs.github.com/rest/reference/repos#create-a-commit-status"
    },
    "message": "Response code 422 (Unprocessable Entity)",
    "stack": "HTTPError: Response code 422 (Unprocessable Entity)\n    at onResponse (/home/ubuntu/renovateapp/node_modules/renovate/node_modules/got/dist/source/as-promise/index.js:142:28)\n    at runMicrotasks ()\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
  }
}

copied from the app.renovatebot.com log output.

To Reproduce

I created a minimal repository, that has that problem: https://github.com/homefully/renovate-bug

Additional context

(My Assumptions!)
It seems like there is a bug in how the github api is used, which causes a 422, which intern gets bubbled up and turned into "repository-changed", which isn't true, it just happens that the bug caused a very similar response code from github.
But having not looked into the renovate code at all, I can only assume that from the logs above.

[viceice]

@rarkins Looks like we don't get the branchCommit because no checkout was done?

https://github.com/renovatebot/renovate/blob/6a0fade7f90daf9861e04f02649ee8a4ca64ed92/lib/platform/github/index.ts#L946-L961

[rarkins]

Is it for a new branch (not on github.com yet) or an existing one? getBranchCommit shouldn't need any checkout

[viceice]

Or we do not update our branch cache corectly here:
https://github.com/renovatebot/renovate/blob/92a26b9b88cb17ba362cfcb86cb04c9ce02f3e7c/lib/util/git/index.ts#L586-L608

[rarkins]

Yes, possible we don't update our cache after a force push? But I don't see that specific detail in the reproduction steps, which implies it happens non stop?

[rarkins]

But isn't this line doing exactly that? https://github.com/renovatebot/renovate/blob/92a26b9b88cb17ba362cfcb86cb04c9ce02f3e7c/lib/util/git/index.ts#L605

[viceice]

maybe something gots wrong and we get 'unknown' as commit?
https://github.com/renovatebot/renovate/blob/92a26b9b88cb17ba362cfcb86cb04c9ce02f3e7c/lib/util/git/index.ts#L589

[reckter]

@rarkins It happens non-stop, (in the reproduction repository, it never even made it to the pull-request stage. The branch is created, but it fails setting the status on that and aborts)

They do exist on github.com in the branches view (https://github.com/homefully/renovate-bug/branches)

[viceice]

lockfile update is missing here:
https://github.com/homefully/renovate-bug/commit/f7805a6db94c0f874fc208ddbfe855b8115c1f46

[reckter]

We noticed, that sometimes the bot manages to update the branches and pr's correctly. Then in all works as intended, and it even manages to auto-merge some branches.
But I have no Idea why it sometimes does work, and at other times it doesn't.

[remal]

I'm suffering from the same issue, GitHub returns sha must be a 40 character SHA1 validation error:

DEBUG: 422 Error thrown from GitHub(branch="renovate/io.grpc-grpc-bom-1.x") { "err": { "name": "HTTPError", "timings": { "start": 1599725600412, "socket": 1599725600412, "lookup": 1599725600414, "connect": 1599725600427, "secureConnect": 1599725600440, "upload": 1599725600440, "response": 1599725600648, "end": 1599725600648, "phases": { "wait": 0, "dns": 2, "tcp": 13, "tls": 13, "request": 0, "firstByte": 208, "download": 0, "total": 236 } }, "message": "Response code 422 (Unprocessable Entity)", "stack": "HTTPError: Response code 422 (Unprocessable Entity)\n at Request. (/home/ubuntu/renovateapp/node_modules/renovate/node_modules/got/dist/source/as-promise/index.js:117:42)\n at runMicrotasks ()\n at processTicksAndRejections (internal/process/task_queues.js:97:5)", "options": { "headers": { "user-agent": "Renovate Bot (GitHub App 2740)", "accept-encoding": "gzip, deflate", "accept": "application/vnd.github.machine-man-preview+json", "authorization": "***********", "content-type": "application/json", "content-length": "164" }, "url": "https://api.github.com/repos/remal/tracing-spec/statuses/6cf9b52", "username": "", "password": "***********", "method": "POST", "http2": false }, "response": { "statusCode": 422, "statusMessage": "Unprocessable Entity", "body": { "message": "Validation Failed", "errors": [ { "resource": "Status", "code": "custom", "field": "sha", "message": "sha must be a 40 character SHA1" } ], "documentation_url": "https://docs.github.com/rest/reference/repos#create-a-commit-status" }, "headers": { "date": "Thu, 10 Sep 2020 08:13:20 GMT", "content-type": "application/json; charset=utf-8", "content-length": "228", "connection": "close", "server": "GitHub.com", "status": "422 Unprocessable Entity", "x-github-media-type": "github.v3; param=machine-man-preview; format=json", "x-ratelimit-limit": "5000", "x-ratelimit-remaining": "4996", "x-ratelimit-reset": "1599729029", "x-ratelimit-used": "4", "access-control-expose-headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset", "access-control-allow-origin": "*", "strict-transport-security": "max-age=31536000; includeSubdomains; preload", "x-frame-options": "deny", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin", "content-security-policy": "default-src 'none'", "vary": "Accept-Encoding, Accept, X-Requested-With", "x-github-request-id": "C74A:3804:AD5F95:D66E2E:5F59E020" } } } }

[viceice]

@rarkins looks like we somehow only get the short commit sha. So this seems to be the root cause of this problem.

I think we need to check why we only get a short sha.

[rarkins]

Flow:

• setStability
• setStatusCheck
• platform.setBranchStatus

And here is where getBranchCommit is called: https://github.com/renovatebot/renovate/blob/5475f42518a56e96b27e86999a618ed5a9493455/lib/platform/github/index.ts#L937

git.getBranchCommit: https://github.com/renovatebot/renovate/blob/5475f42518a56e96b27e86999a618ed5a9493455/lib/util/git/index.ts#L329-L331

It's probably here that we're adding a short branch commit: https://github.com/renovatebot/renovate/blob/5475f42518a56e96b27e86999a618ed5a9493455/lib/util/git/index.ts#L605 (if branch is created or updated on the same run as we set status)

[rarkins]

Maybe we can detect this type of problem if we stringly type shortCommitSha and longCommitSha and pass them around instead of variable length string?

[rarkins]

Perhaps this is a short sha? https://github.com/steveukx/git-js/blob/588b4f8b3e5f98c783cfa04bd0d35eada4c30abd/typings/response.d.ts#L77

[rarkins]

I think I have a fix for this, will push soon

[viceice]

yes, it's because git commit only dumps the short sha, so we should fetch the full sha after commit.

git rev-parse HEAD should print our current commit

[renovate-release]

:tada: This issue has been resolved in version 23.20.6 :tada:

The release is available on:

• GitHub release
• 23.20.6

Your semantic-release bot :package::rocket:

[rarkins]

This will be put into the hosted app later today