Gitea: Seg Fault redux?

[quasiben]

• Gitea version (or commit ref): 1.1.2
• Git version: 2.11.1
• Operating system: Centos 7.3
• Database (use [x]):
  
  
  
  • [x] PostgreSQL
  
  
  • [ ] MySQL
  
  
  • [ ] MSSQL
  
  
  • [ ] SQLite
  
  
• Can you reproduce the bug at https://try.gitea.io:
  
  
  
  • [ ] Yes (provide example URL)
  
  
  • [ ] No
  
  
  • [x] Not relevant
  
  
• Log gist: https://gist.github.com/quasiben/87c130849086510267f252a31455e515
  

  Description

Gitea dies in the middle of starting up. Not exactly sure how to diagnose this issue. Could
issue #1684 be related ?

[quasiben]

I have also tried with 1.1.4

[quasiben]

I'm running a simpler version and I'm seeing a different error:

[root@ap-gitea-2361316594-z8612]# gitea web
2017/10/17 18:36:32 [...s/setting/setting.go:801 NewContext()] [E] Expect user 'root' but current user is:
[root@ap-gitea-2361316594-z8612]# whoami
root

This is running in a Centos 7.3 docker image which I've deployed with Kubernetes. Something wonky appears to be happening with how the current user is being determined here

[lafriks]

User on Linux is determined by checking environment variables USER and USERNAME. Could be that centos does not set them?

[quasiben]

Ah, right. I have a script that was injecting that env var and now I'm back to the original seg fault posted at the top :/

[quasiben]

Switching to 1.2.1 I get the error:

2017/10/17 20:39:24 [I] [SQL] SELECT "user".id FROM "user" WHERE "user".num_repos!=(SELECT COUNT(*) FROM "repository" WHERE owner_id="user".id)
2017/10/17 20:39:24 [...els/issue_indexer.go:89 InitIssueIndexer()] [E] InitIssuesIndexer, open index: error parsing mapping JSON: unexpected end of JSON input
mapping contents:

[pgaskin]

For the 1.2.1 indexer error, you can fix it by deleting the /path/to/gitea/data/dir/indexers/issues.bleve folder and restarting gitea.

For the segfault, try running gitea in gdb, and see if there is a null value in a variable on one of the stack frames (google it, you'll find lots of stuff about debugging segfaults with gdb).

[pgaskin]

According to your stack trace:

/usr/local/go/src/os/user/lookup_unix.go:253 +0x2b fp=0xc42012d4e8 sp=0xc42012d480 os/user.lookupUnixUid(0x0, 0x0, 0x0, 0x0) /usr/local/go/src/os/user/lookup_unix.go:105 +0x161 fp=0xc42012d5b0 sp=0xc42012d4e8 os/user.current(0x0, 0x0, 0xc42013d700) /usr/local/go/src/os/user/lookup_unix.go:49 +0x2f fp=0xc42012d5e0 sp=0xc42012d5b0 os/user.Current(0x131e8e8, 0xa, 0x0)

user.lookupUnixUid seems to be passed nil values for arguments. The code for user.lookupUnixUid is here. user.Current calls syscall.Getuid, which should return an integer with the uid. Instead, it seems to be returning nil. I'm not sure what is going on here more than what I just said. This may somewhat be related to golang/go#13470.

I am not completely certain if this is the issue, as I only occasionally debug segfaults, and it is usually with a debugger like gdb or delve, not by reading stack traces.

[quasiben]

@geek1011, thanks for the comments. I'm pretty sure the issue is as you described -- the user.lookupUnixUid is coming back with nil values. Which version of go was used when building the gitea binary ?

Apologies, I am unfamiliar with golang. I'm trying to build master with golang 1.8.3 but I get an error:

go build -i -v  -tags '' -ldflags '-s -w -X "main.Version=1.1.0+644-g2a184f59" -X "main.Tags="' -o gitea
code.gitea.io/gitea/vendor/github.com/chaseadamsio/goorgeous
code.gitea.io/gitea/modules/markup/orgmode
_/root/gitea
# _/root/gitea
./main.go:40: cannot use cmd.CmdWeb (type "code.gitea.io/gitea/vendor/github.com/urfave/cli".Command) as type "github.com/urfave/cli".Command in array or slice literal

[pgaskin]

@quasiben you need go 1.9.1 to build the latest urfave.cli. I recently ran into this issue myself.

[quasiben]

Was golang 1.8.X used to build binaries ?

[pgaskin]

@quasiben not the most recent ones.

[quasiben]

Ok.

Still seeing errors with 1.9.1

[root@2361316594-z8612 gitea]# go version
go version go1.9.1 linux/amd64
[root@ap-git-storage-2361316594-z8612 gitea]# make
go build -i -v  -tags '' -ldflags '-s -w -X "main.Version=1.1.0+644-g2a184f59" -X "main.Tags="' -o gitea
_/root/gitea
# _/root/gitea
./main.go:40:6: cannot use cmd.CmdWeb (type "code.gitea.io/gitea/vendor/github.com/urfave/cli".Command) as type "github.com/urfave/cli".Command in array or slice literal

But, perhaps upgrading to 1.9.1 won't actually solve my problem here

[quasiben]

I resolved things by setting

export PGPASSFILE="$HOME/.pgpass"

and a colleague submitted an upstream patch: https://github.com/lib/pq/pull/666

[necaris]

Is there a place where this can be documented so that people can find this workaround in future?

[pgaskin]

@quasiben Great job. I guess my analysis of the stack trace was correct.

@necaris This workaround may not be needed in the future because of lib/pq#666

[necaris]

Note that https://github.com/lib/pq/pull/666 has been merged -- does there need to be some kind of govendor update to ensure this is picked up by gitea?

[sapk]

@necaris yes and done (waiting LGTM).