Envoy: PRs with lots of commits will not load in github, showing unicorn instead.

[jmarantz]

Description:
This is really a github issue, but listing it as an Envoy issue because it's affecting our development flow. There is no public github tracker on the github scalability issue, but I thought I'd use this issue to track interaction with github support on it.

Examples of PRs that have exploded are: #2932 and #3059.

[jmarantz]

Original report sent to github:

It looks like after a medium-size PR has enough comments and commits, it unicorns. https://github.com/envoyproxy/envoy/pull/3059 is an example of such a PR.

There is a large number of comments, but not an outrageous number. Is there some scalability issue in github that needs to be sorted out?

[jmarantz]

Response from Tiara ([email protected]):

Hi Joshua,

Thank you for reaching out to GitHub Support, and sorry for the trouble here.

We're currently investigating other similar reports regarding this issue, and I've added your report to our list.

While we can't share a specific ETA just yet, we'll be sure to update you as soon as there's any news to share.

Best regards,
Tiara
GitHub Support

[jmarantz]

Follow-up from me:

Thanks -- is there a github issue about github we could follow?

[jmarantz]

Response from Tiara:

Hi again Joshua,

I'm afraid our issue tracker is internal, so I don't have a public facing issue that you can follow -- but we will definitely update you when we can, though!

If you need to merge that pull request soon, we recommend using git merge locally on the command line as a workaround. We apologize for any inconvenience, and thank you for your patience.

Do let us know if there's anything else we can do to help.

Best regards,
Tiara
GitHub Support

[jmarantz]

Follow-up from me:

Thanks. It would be helpful to know...if you have discovered & can share this...is this a scalability issue related to the commit-count, comment-count, or both?

The workaround you suggested might be helpful at some point, but it's a complicated multi-person project with tons of code-review scrutiny and the person who created this PR does not have write-access; it must be merged by an OWNER. And we haven't really finished iterating on the PR yet. So right now we have a situation where the PR owner has made a new version of the CL to workaround, but that quickly unicorned as well, making me think this was more about the commit-count (he cloned all the commits).

[jmarantz]

Response from John (GitHub Staff):

Hi Joshua,

I'm afraid I'm not able to provide much more detail here, but it does appear that the issue relates to the size of the pull request (you could call this the commit count) but this is not the only factor :)

Our engineers are looking into this though and we'll be sure to let you know once we have any news to share.

Have a great day further!

Cheers,
John
GitHub Support

[mrice32]

I reported this as well, and I got a few slightly different suggestions for those interested:

We're continually seeing a unicorn with the following accompanying text:
"
This page is taking way too long to load.
Sorry about that. Please try refreshing and contact us if the problem persists.
"
The PR in question is this one: https://github.com/envoyproxy/envoy/pull/3059.

We've already closed and reopened this PR once, but again, after a few review cycles, it became unreachable (I've tried refreshing many times throughout the day). Is there anything we can do to preserve our comments and alleviate this issue now and in the future?

Thank you!

Matt Rice

Response:

Hi Matt,

Thanks for writing to GitHub support and sorry for the trouble!

We have received a few similar reports regarding big pull requests. I can confirm that our team is working on fixing this problem, and I've added your report to that issue.

Unfortunately I won't be able to provide an estimated time on when this might get fixed. We know this is not ideal, but hope you understand.

Is there anything we can do to preserve our comments and alleviate this issue now and in the future?

You should be able to use our API to get a copy of the comments in this pull request:

https://developer.github.com/v3/pulls/comments/#list-comments-on-a-pull-request

https://developer.github.com/v3/pulls/reviews/#list-reviews-on-a-pull-request

If you have any question regarding using API we can definitely offer more detailed instructions.

Unfortunately I'm not sure if there's anything you can do on your part to view this pull request normally.

One thing you could try is viewing this PR in Incognito window or logged out.

I was actually able to view the PR you linked because I don't have write access to this repository, so GitHub will try to load less things for me such as edit buttons and so on. Of course this won't allow you to merge the pull request and so on, but perhaps it would help for you to look at its content?

Lastly, while we are working on consistently supporting large pull requests, you might consider keeping pull requests relatively short for now to avoid running into this again.

I hope this helps clear things up, and please let us know if you have any other questions.

Best 🕶
Cici
GitHub Support
👉Find support and come learn with other GitHub users in our new GitHub Community Forum!

[jmarantz]

Follow-up from me:

Can you tell me at least how long this problem has been present? I don't
know if this was always a limitation for github that we've just started
bumping into on a regular basis, or whether this is a transient issue.

The suggested workaround doesn't, as far as I can tell, address the problem
of continuing an ongoing code-review conversation.

We hope a remedy arrives soon.

[jmarantz]

And reply from github:

This is not an expected limitation of our system, though it's hard for us to pinpoint when this exact issue cropped up since we haven't nailed down the specific cause yet.

We've been making improvements to the pull requests feature regularly to handle larger PRs with more commits and more timeline events, like comments and reviews. We also resolved a large number of size-related issues with PRs recently, but as you're experiencing there's more work to do.

The team is actively investigating and we'll follow up as soon as we have news to share.

[ldemailly]

This happens on/off with a number of istio PRs too - for some reason though, using incognito window usually allows to load the content/see it

[mattklein123]

Calling this fixed for now given the recent perf updates shipped by GH.

[potemkin-git]

This issue still exists 2 years later.

Same use case: large PR (273 files, +20k lines, -15k lines), 417 pending comments
Same issue: 504 Bad Gateway on comments POST.

[chloerice]

^Same, I've just pulled an all nighter finishing review of a large PR. It's only got 18 files, but there's a lot of requested changes at over 60 comments and I'm unable to submit it for review 😞 https://github.com/Shopify/polaris-react/pull/2725/files