Gitea: Slow merge with large repos

[twpedersen]

• Gitea version (or commit ref): First seen in 1.9.6, but also exists in 1.10.2
• Git version: 2.22.0
• Operating system: gitea docker image
• Database (use [x]):
  
  
  
  • [ ] PostgreSQL
  
  
  • [ ] MySQL
  
  
  • [ ] MSSQL
  
  
  • [x] SQLite
  
  
• Can you reproduce the bug at https://try.gitea.io:
  
  
  
  • [ ] Yes (provide example URL)
  
  
  • [x] No (clone fails)
  
  
  • [ ] Not relevant
  
  
• Log gist:

Description

note: This issue is not seen in 1.11.0+dev-575-g1e9b3d474

To reproduce:

• push a large repo like https://github.com/torvalds/linux.git into gitea
• open a PR against master, can be a simple change
• try to merge (rebase and merge has same behavior) PR, takes about a minute to complete.

On 1.8.3+2-g11f6ed4f8 the merge takes a few seconds. It seems this issue was originally reported in https://github.com/go-gitea/gitea/issues/601 and https://github.com/go-gitea/gitea/issues/6287 then fixed in https://github.com/go-gitea/gitea/pull/4921 and resurfaced somewhere between 1.8.2 and 1.9.6.

top shows /usr/libexec/git-core/git rev-list --objects --stdin --not --all --quiet using 100% CPU during merge. Generating the commit-graph doesn't help.

[bagasme]

@twpedersen what about your system spec?

[twpedersen]

@bagasme merge takes > 1 minute on a ryzen 3900x. Interestingly it seems to take about the same amount of time on an i3-9100. The git rev-list runs twice for about 30s each, so maybe they're timing out in both cases?

[zeripath]

I suspect the problem is not the merge but at the push in the end - in particular there are a few calls in there to count objects that are quite slow. Others have found this before.

If you can, you could put logging in to test where the problems are.

One other thing that can improve things is ensuring that the git commit graph is built.

[lafriks]

We should probably run commit counting in queue to make it async, that should considerably improve performance

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.