Gitea: /explore/code rendering slowly when there are many repositories (> 2000) and a logged in user

[jnlin]

• Gitea version (or commit ref): 6a1a6332de5b5000d31ec26efaef6f78a4ee10f0
• Git version: 2.11.0
• Operating system: Docker
• Database (use [x]):
  
  
  
  • [ ] PostgreSQL
  
  
  • [X] MySQL
  
  
  • [ ] MSSQL
  
  
  • [ ] SQLite
  
  
• Can you reproduce the bug at https://try.gitea.io:
  
  
  
  • [ ] Yes (provide example URL)
  
  
  • [x] No
  
  
  • [ ] Not relevant
  
  
• Log gist: N/A

Description

We have a lot of repositories (>2000), and the rendering of /explore/code is slow (see the attached screenshot, > 17 seconds) with a logged in user.

We tried to remove some permission checking feature and the speed becomes good, however we are not sure which is the slowest part. The removed code is: https://github.com/go-gitea/gitea/compare/master...jnlin:jnlin_test

The possible parts under suspicion are:
https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330
https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

Screenshots

[lunny]

The differences are GetRepositoriesMapByIDs and ctx.Data["RepoMaps"] = rightRepoMap.

[jnlin]

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330

This spends < 0.5 secs

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

This spends 17 secs, there may be some N+1 queries in CheckUnitUser()

[a1012112796]

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330

This spends < 0.5 secs

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

This spends 17 secs, there may be some N+1 queries in CheckUnitUser()

How about #12854 ?

[jnlin]

How about #12854 ?

Still needs 15 secs

[lunny]

We need more PRs here, but let's merge this one at first.

[a1012112796]

Now the main issue is how to quickly find out all repos that the user has code read permission?

[jnlin]

@a1012112796 yes

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.

[lafriks]

Up