Gitea: panic on self built gitea version v1.0.2

[ston1th]

• Gitea version (or commit ref): v1.0.2
• Operating system: linux amd64

Description

Hello Giteas,

I built the latest gitea version (v1.0.2) and got a panic from the locale library:

panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory

goroutine 1 [running]:
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.initLocales(0xc4202a8278, 0x0, 0xdd2713, 0xb, 0xc420453f20, 0xddc70b, 0x12, 0xc420126160, 0x15, 0x15, ...)
    /home/marius/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:57 +0x5ff
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.I18n(0xc420084b40, 0x1, 0x1, 0x0, 0xc420453f20)
    /home/marius/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:158 +0xee
code.gitea.io/gitea/cmd.newMacaron(0xc42000f040)
    /home/marius/go/src/code.gitea.io/gitea/cmd/web.go:125 +0x715
code.gitea.io/gitea/cmd.runWeb(0xc42000f040, 0x0, 0xc42000f000)
    /home/marius/go/src/code.gitea.io/gitea/cmd/web.go:160 +0x6f
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0xcbdae0, 0xe07d78, 0xc42000f040, 0xc420016900, 0x0)
    /home/marius/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xb9
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0xdc69b6, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0xde1ae2, 0x16, 0x0, ...)
    /home/marius/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xb4b
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc42010a9c0, 0xc42000a160, 0x2, 0x2, 0x0, 0x0)
    /home/marius/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x65f
main.main()
    /home/marius/go/src/code.gitea.io/gitea/main.go:42 +0x377

I think the library (go-macaron/i18n) should handle or return such an error instead of panicking.

Anyway, if I'm using the prebuilt binary (github release page) everything works fine, though.

Thanks,
ston1th

[morrildl]

Building with TAGS="bindata" (to embed assets) might get you past this.

[ston1th]

Thanks, that worked.

Sadly, the panic call in the i18n library, seems unfixable without an API change.

[strk]

Are you saying 1.0.2 needs to be built with TAGS="bindata" or it would panic ? If that's the case this ticket should be reopened and considered high priority

[ston1th]

I don't have a conf/* directory on my server.
The i18n library seems to expect this file (and maybe more files) in there: conf/locale/locale_en-US.ini

So basically you are right, in a setup where this file is missing and v1.0.2 is not built with TAGS="bindata" it will panic.

This line is the problem's source:
https://github.com/go-gitea/gitea/blob/master/vendor/github.com/go-macaron/i18n/i18n.go#L57

In my opinion, a library never should call panic, but return an error.
The problem is, that it seems not fixable without breaking the API, but maybe im wrong there.

[ston1th]

I opened an issue to address this: https://github.com/go-macaron/i18n/issues/5

I didn't test if there are more components affected, when building without bindata.

A quick fix (HEAD at v1.0.2):

diff --git a/cmd/web.go b/cmd/web.go
index 9776fdc1..596f28f4 100644
--- a/cmd/web.go
+++ b/cmd/web.go
@@ -115,14 +115,18 @@ func newMacaron() *macaron.Macaron {
                }
        }

-       m.Use(i18n.I18n(i18n.Options{
+       i18nHandler, err := i18n.I18n(i18n.Options{
                SubURL:      setting.AppSubURL,
                Files:       localFiles,
                Langs:       setting.Langs,
                Names:       setting.Names,
                DefaultLang: "en-US",
                Redirect:    true,
-       }))
+       })
+       if err != nil {
+               log.Fatal(4, "Failed to initialize locale middleware: %v", err)
+       }
+       m.Use(i18nHandler)
        m.Use(cache.Cacher(cache.Options{
                Adapter:       setting.CacheAdapter,
                AdapterConfig: setting.CacheConn,
diff --git a/vendor/github.com/go-macaron/i18n/i18n.go b/vendor/github.com/go-macaron/i18n/i18n.go
index 3b5b1b83..cf795d57 100644
--- a/vendor/github.com/go-macaron/i18n/i18n.go
+++ b/vendor/github.com/go-macaron/i18n/i18n.go
@@ -33,7 +33,7 @@ func Version() string {
 }

 // initLocales initializes language type list and Accept-Language header matcher.
-func initLocales(opt Options) language.Matcher {
+func initLocales(opt Options) (language.Matcher, error) {
        tags := make([]language.Tag, len(opt.Langs))
        for i, lang := range opt.Langs {
                tags[i] = language.Raw.Make(lang)
@@ -54,10 +54,10 @@ func initLocales(opt Options) language.Matcher {

                err := i18n.SetMessageWithDesc(lang, opt.Names[i], locale, custom...)
                if err != nil && err != i18n.ErrLangAlreadyExist {
-                       panic(fmt.Errorf("fail to set message file(%s): %v", lang, err))
+                       return nil, err
                }
        }
-       return language.NewMatcher(tags)
+       return language.NewMatcher(tags), nil
 }

 // A Locale describles the information of localization.
@@ -153,9 +153,12 @@ type LangType struct {
 // I18n is a middleware provides localization layer for your application.
 // Paramenter langs must be in the form of "en-US", "zh-CN", etc.
 // Otherwise it may not recognize browser input.
-func I18n(options ...Options) macaron.Handler {
+func I18n(options ...Options) (macaron.Handler, error) {
        opt := prepareOptions(options)
-       m := initLocales(opt)
+       m, err := initLocales(opt)
+       if err != nil {
+               return nil, err
+       }
        return func(ctx *macaron.Context) {
                isNeedRedir := false
                hasCookie := false
@@ -221,5 +224,5 @@ func I18n(options ...Options) macaron.Handler {
                if opt.Redirect && isNeedRedir {
                        ctx.Redirect(opt.SubURL + ctx.Req.RequestURI[:strings.Index(ctx.Req.RequestURI, "?")])
                }
-       }
+       }, nil
 }

Another (ugly) "fix" would be to use recover:

package main

import (
        "fmt"
)

func willPanic() int {
        panic("panic")
        return 1
}

func panicWrapper(err *error) int {
        defer func() {
                *err = fmt.Errorf("%v", recover())
        }()
        return willPanic()
}

func main() {
        var err error
        i := panicWrapper(&err)
        fmt.Println(i, err)
}

[ston1th]

Following Unknown's response (https://github.com/go-macaron/i18n/issues/5) this will not be fixed in the main repository.

To avoid the ugly workarounds obove, I would suggest to create a fork of https://github.com/go-macaron/i18n, break the API and fix the problem in the library itself.

What do you guys think?

[lunny]

@tboerger any idea?

[tboerger]

Just set https://github.com/go-macaron/i18n/blob/master/i18n.go#L78 properly on our side.

[ston1th]

But, this would not prevent the application to panic on file system errors per se?

For example:
If the directory/file has wrong access priviledges or they are missing, than gitea would panic (and same as in my case no error would be written to the logs).

[tboerger]

But, this would not prevent the application to panic on file system errors per se?

For example:
If the directory/file has wrong access priviledges or they are missing, than gitea would panic (and same as in my case no error would be written to the logs).

Of course not, but we can not swap everything directly.

[klingtnet]

I get the same error for gitea build from the latest master with the bindata tag enabled.

$ ./gitea web
2017/04/17 16:19:47 [W] Custom config '/tmp/custom/conf/app.ini' not found, ignore this if you're running first time
2017/04/17 16:19:47 [T] Custom path: /tmp/custom
2017/04/17 16:19:47 [T] Log path: /tmp/log
2017/04/17 16:19:47 [I] Gitea v1.1.0+dev
2017/04/17 16:19:47 [I] Log Mode: Console(Trace)
2017/04/17 16:19:47 [I] XORM Log Mode: Console(Trace)
2017/04/17 16:19:47 [I] Cache Service Enabled
2017/04/17 16:19:47 [I] Session Service Enabled
2017/04/17 16:19:47 [I] Run Mode: Development
panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory

[cez81]

@klingtnet If it was built with bindata it should say so in the version tag. How did you build it?

2017/04/17 13:36:41 [I] Gitea v1.1.0+122-gc764a54 built with: bindata, sqlite

[klingtnet]

go get -d -u -v code.gitea.io/gitea && go get -u github.com/jteeuwen/go-bindata/...
pushd $GOPATH/src/code.gitea.io/gitea
PATH=$GOPATH/bin:$PATH TAGS="bindata" make generate build

[klingtnet]

I am compiling with:

$  go version  
go version go1.8.1 linux/amd64

[klingtnet]

I was able to solve the issue in the meantime, the problem was on my side.
Instead of using the gitea binary in $GOPATH/src/code.gitea.io/gitea the one from $GOPATH/bin is used that did not include the bindata stuff.
Maybe the Makefile's build target should copy the binary into $GOPATH/bin, too?

[ptman]

can this be reproduced using latest stable? 1.2.1

[lafriks]

Closing issue, if there is still a problem please reopen

[disassembler]

FYI, I ran into the same issue building this for NixOS. Work around for me was copying locale directory into the build output and then in the prestart systemd service script if it doesn't exist already, I copy the locale dir into conf. Related to https://github.com/NixOS/nixpkgs/pull/30528

[taskforce88]

I have tried to install gitea on a freenas jail.

I used these commands...

pkg install gitea
cd /usr/ports/www/gitea
make install clean
..then I have tried to start with
gitea web
and got the same issue

`root@gitea_1:/usr/ports/www/gitea/files # gitea web
2017/10/19 23:33:23 [T] Custom path: /usr/local/sbin/custom
2017/10/19 23:33:23 [T] Log path: /usr/local/sbin/log
2017/10/19 23:33:23 [I] Gitea v1.2.1
2017/10/19 23:33:23 [I] Log Mode: Console(Trace)
2017/10/19 23:33:23 [I] XORM Log Mode: Console(Trace)
2017/10/19 23:33:23 [I] Cache Service Enabled
2017/10/19 23:33:23 [I] Session Service Enabled
2017/10/19 23:33:23 [I] SQLite3 Supported
2017/10/19 23:33:23 [I] Run Mode: Development
panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory

goroutine 1 [running]:
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.initLocales(0xc420012315, 0x0, 0x116dbaa, 0xb, 0xc420462ab0, 0x11789f5, 0x12, 0xc4200cab00, 0x15, 0x15, ...)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:57 +0x7fc
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.I18n(0xc42014c240, 0x1, 0x1, 0x0, 0xc420462ab0)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:158 +0xcb
code.gitea.io/gitea/routers/routes.NewMacaron(0xc420218140)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/routers/routes/routes.go:93 +0x74c
code.gitea.io/gitea/cmd.runWeb(0xc420218140, 0x0, 0x0)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/cmd/web.go:62 +0xba
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0xffd8c0, 0x11b1658, 0xc420218140, 0xc42028e100, 0x0)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xb7
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x1162305, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x117e955, 0x16, 0x0, ...)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xa72
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc4204b4000, 0xc42000c060, 0x2, 0x2, 0x0, 0x0)
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x5ff
main.main()
/usr/ports/www/gitea/work/src/code.gitea.io/gitea/main.go:44 +0x3a5
root@gitea_1:/usr/ports/www/gitea/files #
`
Can anyone help me to install this package?

[iou1name]

Ditto. I'm running a fresh FreeNAS jail (FreeBSD 11.0-STABLE) and installed gitea from ports with make install clean, and anytime I run gitea web (as root, user git or any other user) I get:

[git@gitea1 ~]$ gitea web
2017/10/25 17:49:16 [T] Custom path: /usr/local/sbin/custom
2017/10/25 17:49:16 [T] Log path: /usr/local/sbin/log
2017/10/25 17:49:16 [I] Gitea v1.2.1
2017/10/25 17:49:16 [I] Log Mode: Console(Trace)
2017/10/25 17:49:16 [I] XORM Log Mode: Console(Trace)
2017/10/25 17:49:16 [I] Cache Service Enabled
2017/10/25 17:49:16 [I] Session Service Enabled
2017/10/25 17:49:16 [I] SQLite3 Supported
2017/10/25 17:49:16 [I] Run Mode: Development
panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory

goroutine 1 [running]:
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.initLocales(0xc420260db5, 0x0, 0x115edea, 0xb, 0xc420489740, 0x1169c35, 0x12, 0xc4200e4dc0, 0x15, 0x15, ...)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:57 +0x7fc
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.I18n(0xc420164540, 0x1, 0x1, 0x0, 0xc420489740)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:158 +0xcb
code.gitea.io/gitea/routers/routes.NewMacaron(0xc4200e6000)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/routers/routes/routes.go:93 +0x74c
code.gitea.io/gitea/cmd.runWeb(0xc4200e6000, 0x0, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/cmd/web.go:62 +0xba
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0xfeeb00, 0x11a2898, 0xc4200e6000, 0xc4202bc900, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xb7
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x1153545, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x116fb95, 0x16, 0x0, ...)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xa72
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc420084820, 0xc42008e000, 0x2, 0x2, 0x0, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x5ff
main.main()
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/main.go:44 +0x3a5

[GlenHertz]

I'm on FreeBSD 11.1 and get the same issue:

2017/11/13 03:25:27 [T] Custom path: /usr/local/sbin/custom
2017/11/13 03:25:27 [T] Log path: /usr/local/sbin/log
2017/11/13 03:25:27 [I] Gitea v1.2.3
2017/11/13 03:25:27 [I] Log Mode: Console(Trace)
2017/11/13 03:25:27 [I] XORM Log Mode: Console(Trace)
2017/11/13 03:25:27 [I] Cache Service Enabled
2017/11/13 03:25:27 [I] Session Service Enabled
2017/11/13 03:25:27 [I] SQLite3 Supported
2017/11/13 03:25:27 [I] Run Mode: Development
panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory

goroutine 1 [running]:
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.initLocales(0xc4202252f5, 0x0, 0x1161240, 0xb, 0xc4202603c0, 0x116c079, 0x12, 0xc4200b8dc0, 0x15, 0x15, ...)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:57 +0x7fc
code.gitea.io/gitea/vendor/github.com/go-macaron/i18n.I18n(0xc420136180, 0x1, 0x1, 0x0, 0xc4202603c0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/go-macaron/i18n/i18n.go:158 +0xcb
code.gitea.io/gitea/routers/routes.NewMacaron(0xc42041fe00)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/routers/routes/routes.go:93 +0x74c
code.gitea.io/gitea/cmd.runWeb(0xc42041fe00, 0x0, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/cmd/web.go:62 +0xba
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0xff0cc0, 0x11a4d00, 0xc42041fe00, 0xc420426a00, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xb7
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x11559a5, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1171fd9, 0x16, 0x0, ...)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xa72
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc420456340, 0xc42000c060, 0x2, 0x2, 0x0, 0x0)
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x5ff
main.main()
        /usr/ports/www/gitea/work/src/code.gitea.io/gitea/main.go:44 +0x3a5

I tried with setenv TAGS bindata and recompiling go and gitea without any luck (same error).

Anyone have a more detailed set of workaround instructions?

[ston1th]

@GlenHertz I just tried to build the latest release version, this is how it worked:

$ make generate
$ TAGS=bindata make

[mqudsi]

I can confirm, it seems that there is a separate issue that is affecting FreeBSD users. @ston1th you're not on FreeBSD, are you?

[ston1th]

@mqudsi no, I'm on linux. So my proposed fix did not work for you?

[mqudsi]

Nope. It actually is not related to FreeBSD, though (what an odd coincidence looking at the comments here, though).

It turns out that gogs supports a translation that gitea does not. There is no such thing as gl-ES for gitea, but if you are importing from gogs, your app.ini will include that translation, causing the panic.

[tnt]

Building on CenOS 7 I got the same message when I ran ./gitea web directly after make generate build. Doing ln -s options conf fixed it for running it from the build dir. In an automatically created custom/conf/ dir (created when I tried to run it from /var/lib/gitea which is suggested somewhere in the docs) it did not find the files that I had copied to /var/lib/gitea/custom/conf/locale. I again had to create a symlink (/var/lib/gitea/conf -> custom/conf).

Now it finally running but not on the port that set in /etc/gitea.ini (which is apparently completely ignored) and I am getting a 500: "html/template: "home" is undefined".

Why can't you just put one single example of where to put which files into the docs?

"A painless self-hosted Git service"? What I am experiencing is the opposite of painless.

[lafriks]

@tnt please follow docs on how to build gitea: https://docs.gitea.io/en-us/install-from-source/

From docs page on how gitea should be built:

TAGS="bindata" make generate build

If you would have done so there would not be no need to do any symlinks etc as all required resources would be included in binary and everything would work out of box

[tnt]

After some additional adjustments setting up the database was admittedly absolutely painless.

[tnt]

@lafriks Your right. I had omitted the bindata tag because i suspected that this maybe prevented some settings. In the meantime I already compiled it with that tag again which is supposedly one of the preconditions that made it finally work.

[dansbandit]

To @GlenHertz and all the other FreeBSD users, I had the same problem when starting gitea with gitea web both by installing with pkg install gitea and ports install. But if you start it with the service gitea start it works. The config file is in /usr/local/etc/gitea/conf/app.ini

[BrotherWarrior]

Nope. It actually is not related to FreeBSD, though (what an odd coincidence looking at the comments here, though).

It turns out that gogs supports a translation that gitea does not. There is no such thing as gl-ES for gitea, but if you are importing from gogs, your app.ini will include that translation, causing the panic.

Please!!! Add this answer to Troubleshooting section of migration from gogs to gitea topic (https://docs.gitea.io/en-us/upgrade-from-gogs/). You need to change settings in app.ini and remove unsupported gogs translations or add translations manually.

I've lost about 2 hours.

[zeripath]

We probably just need to detect the available translations and drop this stupid config or just use it to restrict what translations are available. Similarly we should allow people to override individual translation files with custom translations but back fill into the inbuilt translations and we should use the hierarchy of languages properly so that if you want to localise you only have to change the values that you need to.

[zeripath]

But what we certainly shouldn't do is panic because some one has made a tiny mistake in their config. We have so many log levels, surely that's just a Warn?!

[lunny]

@zeripath I like your idea to detect if a translation is available and I think we have supported user customized locale files to put them on custom/locales directory.

[zeripath]

Ah but the custom locales files have to be complete so if you want to change just one translation you need the whole file. It should fallback to the standard one.

I'm terms of hierarchy, in Java if wanted to provide Hispanic vs Mainland Spanish translations I would look at the codes, and decide that although canonical Spanish as defined by the real academia española is es_ES if I put the majority of my translation in to es with the Hispanic variants in es and the European variants in to es-ES then I can keep the es-ES file very small. We can then use the hierarchy to provide local variants such as en-GB_Company or domain, to change terms to local things.

[athompso]

FWIW, this also affects the prebuilt binaries in OpenBSD 6.6. Per the suggestion in #535 I've tried copying the "options" folder around to a few different places, but all result in the same failure. (How does one tell what, exactly, the "installation folder" is when using a prebuilt binary from packages?)

EDIT: from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238230 I tried setting GITEA_WORK_DIR and GITEA_CUSTOM and that let me start gitea as root.

[aureooms]

Same issue after an Arch update.

[techknowlogick]

Locking this ticket as it has since been closed, and refers to a long outdated version. Please open a new ticket for issues similar to this.