Hugo: Panic on zero-padded taxonomy weights

[clorenz]

$ hugo server -D --debug

[...]
DEBUG 2018/04/15 18:49:36 Load Data from 1 source(s)
INFO 2018/04/15 18:49:36 found taxonomies: map[string]string{"category":"categories", "manufacturerletter":"manufacturerletters", "manufacturer":"manufacturers", "featured":"featured", "tag":"tags"}
panic: interface conversion: interface {} is float64, not int

goroutine 50 [running]:

github.com/gohugoio/hugo/hugolib.(*Site).assembleTaxonomies(0xc4204ce2c0)
    /go/src/github.com/gohugoio/hugo/hugolib/site.go:1545 +0xee2
github.com/gohugoio/hugo/hugolib.(*Site).buildSiteMeta(0xc4204ce2c0, 0x0, 0x0)
    /go/src/github.com/gohugoio/hugo/hugolib/site.go:1367 +0x83
github.com/gohugoio/hugo/hugolib.(*HugoSites).assemble(0xc4205dab80, 0xc421824300, 0x0, 0x0)
    /go/src/github.com/gohugoio/hugo/hugolib/hugo_sites_build.go:177 +0x19a
github.com/gohugoio/hugo/hugolib.(*HugoSites).Build(0xc4205dab80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /go/src/github.com/gohugoio/hugo/hugolib/hugo_sites_build.go:60 +0x144
github.com/gohugoio/hugo/commands.(*commandeer).buildSites(0xc42013a0c0, 0x0, 0x0)
    /go/src/github.com/gohugoio/hugo/commands/hugo.go:827 +0x8b
github.com/gohugoio/hugo/commands.(*commandeer).fullBuild.func3(0x8, 0xed1560)
    /go/src/github.com/gohugoio/hugo/commands/hugo.go:423 +0x2e
github.com/gohugoio/hugo/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1(0xc4205da580, 0xc4206603d0)
    /go/src/github.com/gohugoio/hugo/vendor/golang.org/x/sync/errgroup/errgroup.go:58 +0x57
created by github.com/gohugoio/hugo/vendor/golang.org/x/sync/errgroup.(*Group).Go
    /go/src/github.com/gohugoio/hugo/vendor/golang.org/x/sync/errgroup/errgroup.go:55 +0x66

I have no clue, what might be the reason for this panic. My site currently has got 1545 .md files, maybe it has surpassed an internal limit? Any idea, how to get rid of it?

Thank you very much in advance
Christoph

[kaushalmodi]

Can you post your site source?

[anthonyfok]

And please tell us the Hugo version you are using, and from where you obtain it, and on which OS you are running, etc. Many thanks!

[clorenz]

I'm using the official version, downloaded from github,

Hugo Static Site Generator v0.38.2 linux/amd64 BuildDate: 2018-04-09T08:17:17Z

on an Ubuntu 16.04 LTS.

The content (~1250 pages) is attached, as well as my config.toml

content.tar.gz

config.toml.gz

Thank you very much in advance

[anthonyfok]

Thank you @clorenz.

I was able to reproduce the panic on my Debian sid (amd64) running hugo 0.39-1 compiled on Debian.
(will be available via apt install hugo later today or tomorrow). Not sure exactly why though. Will need to investigate.

[moorereason]

I'm sure the panic is the result of two unprotected typecasts here and here. I'm not sure what's causing it, though. I don't see where the float64 is coming from.

[kaushalmodi]

Is this a problem:

km²~/temp/:content.1> rg ':\s[0-9]{7,}(\.[0-9]+)*$'
movements/r/rego/rego-2125-216.de.md
15:manufacturers_weight: 2125216

movements/r/rego/rego-2125-216.en.md
15:manufacturers_weight: 2125216

movements/f/fe/fe-4611.en.md
15:manufacturers_weight: 4611000

movements/f/fe/fe-4611.de.md
15:manufacturers_weight: 4611000

I don't know when Go assigns values as float64 instead of int.

I see some absurdly high taxonomy weights there.. but they are not as high as 2^31 = 2147483648. So not sure..

[kaushalmodi]

@clorenz It would be wayy easier to reproduce your issue if you simply archived everything needed to build the site.. Only content and config is not enough, you need layouts or theme to build too.

---

Building sites … ERROR 2018/04/17 18:23:27 Unable to locate template for shortcode "movementlist" in page "movements/u/urofa/_index.en.md"
ERROR 2018/04/17 18:23:27 Unable to locate template for shortcode "movementlist" in page "movements/u/urofa/_index.de.md"

[kaushalmodi]

OK, so my theory is correct.. there's a problem with one of the values of manufacturers_weight.. because if I delete all manufacturers_weight lines, the site builds fine:

km²~/temp/:hugo_temp/content> \find . -name "*.md" -print0 | xargs -0 \sed -i '/manufacturers_weight:.*/d'                      04/17 6:34pm
km²~/temp/:hugo_temp/content> ..                                                                                                04/17 6:36pm
km²~/temp/:hugo_temp> srvh -t bare-min -p 5656 
                   |  DE  |  EN
+------------------+------+------+
  Pages            | 1498 | 1490
  Paginator pages  |   63 |   62
  Non-page files   |    0 |    0
  Static files     |    2 |    2
  Processed images |    0 |    0
  Aliases          |    1 |    1
  Sitemaps         |    1 |    1
  Cleaned          |    0 |    0

Total in 12105 ms
Watching for changes in /home/kmodi/temp/hugo_temp/{content,layouts,themes}
Watching for config changes in /home/kmodi/temp/hugo_temp/config.toml
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:5656/ (bind address 127.0.0.1)
Web Server is available at http://localhost:5657/ (bind address 127.0.0.1)
Press Ctrl+C to stop

[kaushalmodi]

Not at my PC now.. but I just realized.. do any of those weight values have a dot or a comma or an exponent sign like e or E?

[kaushalmodi]

Got bit by this again.. some genius in the Go lang team thought that it's a good idea to consider string representations of numbers starting with 0 as octals!

So the temporary user-facing fix is:

find . -name "*.md" -exec grep -P 'manufacturers_weight: 0[0-9]+' -l {} \; -exec \sed -r -i 's/(manufacturers_weight: )0([0-9]+)/\1\2/' {} \;

• Removes preceding zeros from all values of manufacturers_weight.

You will end up with these 16 modified files.

---

Though it's not clear why strings considered as octal representations are seen as float64.

And the fix on Hugo side would be catch this exception and point to the file/line containing the offending uncastable int-string.

[clorenz]

After changing the manufacturers_weight items to contain no leading zeroes, the panic is gone.

Thank you very much for the great support!

[kaushalmodi]

@bep @moorereason Does the suggestion made here make sense in this case.. to use strconv.Atoi() or explicitly setting the base to 10 for strconv.ParseInt().. because we don't expect the weights to be octal/hex/..?

[moorereason]

@kaushalmodi,
My understanding of the situation is that we're retrieving a page parameter, and the result is a float64. The type of value we get back will depend on the library we're using for the front matter parsing (YAML vs TOML vs JSON).

I would fix the two places I linked to above with cast.ToIntE to force the value into an int.