Gitea: Indexer crashes

[gerroon]

Indexer crashes with the lines below

I am using Gitea Version: 34695f4 on Debian Testing

2018/11/12 11:37:06 [.../runtime/asm_amd64.s:1333 goexit()] [E] updateRepoIndexer: exit status 128 - fatal: git cat-file: could not get object info

2018/11/12 11:37:06 [I] Done populating the repo indexer with existing repositories
2018/11/12 11:37:07 [.../runtime/asm_amd64.s:1333 goexit()] [E] updateRepoIndexer: exit status 128 - fatal: Not a valid object name 33af0e1f6c425959dd41b96752e61f50949e0118
57c601d489a0901efaa5ded23b724bd5fb7f5af2

2018/11/12 11:37:22 [.../runtime/asm_amd64.s:1333 goexit()] [E] updateRepoIndexer: exit status 128 - fatal: Not a valid object name d23a53c74af7601d45f90b182bcde035ef0d691c
6ddd2549edfd65ec21fe54efae21e963b83fba1e

[techknowlogick]

Which version of Git are you using?

[gerroon]

git --version
git version 2.19.1

Debian Buster X64

[tYYGH]

I have a similar problem: same error message, same status 128. And Gitea is then taking up 44% of my whole RAM!
This is after changing from:

• git 2.18.0(-1)
• gitea 1.5.1(-1)

to:

• git 2.19.1(-1)
• gitea 1.5.3(-3)

on Archlinux, with my Ansible configuration applied to app.ini.sample from upstream Gitea.

Please ask for details if you need more.

[xoxys]

Same here on RHEL:

updateRepoIndexer: exit status 128 - error: unable to find XXX fatal: git cat-file XXX: bad file

• git version 1.8.3.1
• gitea 1.5.3

[MarkusAmshove]

Same after updating from 1.5.1 to 1.6.0:

• SLES 12
• git version 2.12.3
• gitea 1.6.0

[laoshaw]

Same, using 1.6.0, debian 9 64bit, git 2.17.1

2018/11/30 22:56:10 [.../runtime/asm_amd64.s:1333 goexit()] [E] updateRepoIndexer: exit status 128 - fatal: git cat-file 9d7337a3b028095abbb0ee3dec460544d019d1f6: bad file

[ll911]

same, using 1.6.1, rhel 7.5 X86_64bit, git 2.14.1

2018/12/17 15:06:09 [...routers/repo/http.go:391 serviceRPC()] [E] fail to serve RPC(upload-pack): exit status 128 - {[102 97 116 97 108 58 32 84 104 101 32 114 101 109 111 116 101 32 101 110 100 32 104 117 110 103 32 117 112 32 117 110 101 120 112 101 99 116 101 100 108 121 10] 0 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 0}
2018/12/17 17:29:40 [.../runtime/asm_amd64.s:1333 goexit()] [E] updateRepoIndexer: exit status 128 - fatal: git cat-file: could not get object info

[MarkusAmshove]

This is still present in 1.7 for me

[gerroon]

The indexer seems to be a bit useless with big repos :(

[tYYGH]

As things stand, it is _also_ useless with little repos, because you have to disable it to keep it from eating all your RAM :-/

[laoshaw]

indexing and searching are essential to code related work, any way to improve this in gitea, as far as I can tell, this is the only obvious weak area at the moment.

[lunny]

@laoshaw Yes, that needs more work.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

[tYYGH]

Does someone know the current status of this issue?
Also, what are the consequences of switching the indexer off? I wish to understand if this issue is minor or major, as far as I am concerned at least…

[apricote]

@tYYGH If you switch the Repo Indexer off, you will not be able to search for code in Gitea anymore.

[gerroon]

I am wondering about the indexer situation as well. Has there been any improvements?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

[gerroon]

@stale you suck

[TBK]

Here is a full Drone CI Cloud buildlog running timeout -s ABRT 20m make test-sqlite documenting the issue - https://cloud.drone.io/alpinelinux/aports/7389/3/1

[6543]

is this fixed now?

because it looks like it is fixed in 1.9.2 (https://cloud.drone.io/alpinelinux/aports/11110/3/1)

[sapk]

I will have a look but I remember a PR for a buffered git cat-file implementation. And some code as migrated from calling git command to go-git lib.

[guillep2k]

@sapk @6543 It's probably fixed in #7711 and others.

[sapk]

@guillep2k Yes, you greatly improve indexer code and others have greatly contribute to git related performances.

I will close this issue then.
Please feel free to re-open an issue (and link this one) if the problem is still existing.