Hugo: Slash issue in themesDir and similar settings

Created on 7 Jun 2017 · 22Comments · Source: gohugoio/hugo

C:\Users\xmr\Desktop\foo-bar>hugo version
Hugo Static Site Generator v0.21 windows/amd64 BuildDate: 2017-06-05T19:44:37+03:00

C:\Users\xmr\Desktop\foobar>hugo server --watch --verbose
INFO 2017/06/07 21:53:42 Using config file:
INFO 2017/06/07 21:53:42 C:\Users\xmr\Desktop\foobar\src\static\ is the only static directory available to sync from
INFO 2017/06/07 21:53:42 syncing static files to C:\Users\xmr\Desktop\foobar\
WARN 2017/06/07 21:53:42 No translation bundle found for default language "en"
WARN 2017/06/07 21:53:42 Translation func for language en not found, use default.
WARN 2017/06/07 21:53:42 i18n not initialized, check that you have language file (in i18n) that matches the site language or the default language.
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/spf13/hugo/tpl/tplimpl.(*templateHandler).loadTemplates.func1(0xc0421aeb90, 0x4a, 0xf33f40, 0xc0421bd740, 0x0, 0x0, 0xc04244e580, 0xc04244e560)
        /Users/bep/go/src/github.com/spf13/hugo/tpl/tplimpl/template.go:448 +0xd35
github.com/spf13/hugo/vendor/github.com/spf13/afero.walk(0xf359a0, 0xfa7d40, 0xc0421aeb90, 0x4a, 0xf33f40, 0xc0421bd740, 0xc042450780, 0x0, 0x0)
        /Users/bep/go/src/github.com/spf13/hugo/vendor/github.com/spf13/afero/path.go:44 +0x88
github.com/spf13/hugo/vendor/github.com/spf13/afero.walk(0xf359a0, 0xfa7d40, 0xc042452040, 0x34, 0xf33f40, 0xc0421bd5c0, 0xc042450780, 0x0, 0x34)
        /Users/bep/go/src/github.com/spf13/hugo/vendor/github.com/spf13/afero/path.go:69 +0x462
github.com/spf13/hugo/vendor/github.com/spf13/afero.Walk(0xf359a0, 0xfa7d40, 0xc042452040, 0x34, 0xc042450780, 0x6, 0x64)
        /Users/bep/go/src/github.com/spf13/hugo/vendor/github.com/spf13/afero/path.go:107 +0x10a
github.com/spf13/hugo/helpers.SymbolicWalk(0xf359a0, 0xfa7d40, 0xc042450750, 0x2b, 0xc042450780, 0x2, 0x2b)
        /Users/bep/go/src/github.com/spf13/hugo/helpers/path.go:507 +0x2f6
github.com/spf13/hugo/tpl/tplimpl.(*templateHandler).loadTemplates(0xc042352800, 0xc042450750, 0x2b, 0x0, 0x0)
        /Users/bep/go/src/github.com/spf13/hugo/tpl/tplimpl/template.go:480 +0x207
github.com/spf13/hugo/tpl/tplimpl.(*templateHandler).LoadTemplates(0xc042352800, 0xc042450750, 0x2b, 0x0, 0x0)
        /Users/bep/go/src/github.com/spf13/hugo/tpl/tplimpl/template.go:290 +0x5a
github.com/spf13/hugo/hugolib.(*Site).withSiteTemplates.func1(0xf344e0, 0xc042352800, 0x0, 0xfa8070)
        /Users/bep/go/src/github.com/spf13/hugo/hugolib/hugo_sites.go:136 +0x87
github.com/spf13/hugo/tpl/tplimpl.(*TemplateProvider).Update(0x0, 0xc042160750, 0x0, 0x0)
        /Users/bep/go/src/github.com/spf13/hugo/tpl/tplimpl/templateProvider.go:35 +0x4cb
github.com/spf13/hugo/deps.(*Deps).LoadResources(0xc042160750, 0xc04200d580, 0xc0421bc240)
        /Users/bep/go/src/github.com/spf13/hugo/deps/deps.go:68 +0x79
github.com/spf13/hugo/hugolib.applyDepsIfNeeded(0xc0420560e0, 0xc04200d580, 0xc0421bc240, 0xf346c0, 0xc042133680, 0xf2d260, 0x0, 0xc0421b06f0, 0xf2d1e0, 0xc0421d0740, ...)
        /Users/bep/go/src/github.com/spf13/hugo/hugolib/hugo_sites.go:107 +0x308
github.com/spf13/hugo/hugolib.newHugoSites(0xc0420560e0, 0xc04200d580, 0x0, 0xf346c0, 0xc042133680, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/bep/go/src/github.com/spf13/hugo/hugolib/hugo_sites.go:66 +0x2ed
github.com/spf13/hugo/hugolib.NewHugoSites(0xc0420560e0, 0xc04200d580, 0x0, 0xf346c0, 0xc042133680, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/bep/go/src/github.com/spf13/hugo/hugolib/hugo_sites.go:131 +0xd5
github.com/spf13/hugo/commands.(*commandeer).initSites(0xc0421d18c0, 0x17, 0x72)
        /Users/bep/go/src/github.com/spf13/hugo/commands/hugo.go:741 +0x5d
github.com/spf13/hugo/commands.(*commandeer).buildSites(0xc0421d18c0, 0x1, 0x0, 0xc042176a20)
        /Users/bep/go/src/github.com/spf13/hugo/commands/hugo.go:752 +0x39
github.com/spf13/hugo/commands.(*commandeer).build(0xc0421d18c0, 0xc04202dade, 0x1, 0x1, 0xc0421d4610, 0x16)
        /Users/bep/go/src/github.com/spf13/hugo/commands/hugo.go:529 +0xae
github.com/spf13/hugo/commands.server(0xf83ac0, 0xc042002ca0, 0x0, 0x2, 0x0, 0x0)
        /Users/bep/go/src/github.com/spf13/hugo/commands/server.go:162 +0x840
github.com/spf13/hugo/vendor/github.com/spf13/cobra.(*Command).execute(0xf83ac0, 0xc042002c60, 0x2, 0x2, 0xf83ac0, 0xc042002c60)
        /Users/bep/go/src/github.com/spf13/hugo/vendor/github.com/spf13/cobra/command.go:620 +0x3ff
github.com/spf13/hugo/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xf82200, 0xbfe518, 0x4043d3, 0xc042014178)
        /Users/bep/go/src/github.com/spf13/hugo/vendor/github.com/spf13/cobra/command.go:699 +0x340
github.com/spf13/hugo/commands.Execute()
        /Users/bep/go/src/github.com/spf13/hugo/commands/hugo.go:172 +0x67
main.main()
        /Users/bep/go/src/github.com/spf13/hugo/main.go:27 +0x3d

archetypedir:       "src/archetypes"
contentdir:         "src/content"
dataDir:            "src/data"
layoutdir:          "src/layouts"
staticdir:          "src/static"
themesdir:          "src/themes"

The culprit seems to be the src directory; if I move the folders one dir up, to the same dir as config.yml then everything works.

It's like Hugo can't figure out the config file on Windows. Because I still don't get the config path after Using config file: even when things work.

OS: Windows 7 64-bit.

Bug

Source

XhmikosR

👍3

Most helpful comment

Having the exact same issue here! I'm using the same version of hugo with the identical configuration (yet as JSON), but on Windows 10 64 bit.

However, I've found a fix: Changing the forward slashes to backward flashes seems to fix the issue. So, for example, changing src/content to src\\content (double slash because of escaping in JSON) makes Hugo build the site properly.

Still - is this intended? Seems to me it's a Windows-only issue related of not normalizing paths correctly?

dominique-mueller on 8 Jun 2017

👍2 ❤1 😄1

All 22 comments

Having the exact same issue here! I'm using the same version of hugo with the identical configuration (yet as JSON), but on Windows 10 64 bit.

Still - is this intended? Seems to me it's a Windows-only issue related of not normalizing paths correctly?

dominique-mueller on 8 Jun 2017

👍2 ❤1 😄1

Still - is this intended?

No, it is a bug and we will fix it. But remember that Hugo is all about paths and URLs and we support Windows + a bunch of Unix variants, so this is an easy mistake to make.

But a change to src\\content should be future safe and should in my head also work fine on *nix.

bep on 8 Jun 2017

👍1

No worries, it's easy to miss something.

Are there any real tests? If so you could use AppVeyor for Windows testing.

EDIT: I see you already use AppVeyor so I guess it's something the tests don't cover :)

XhmikosR on 8 Jun 2017

EDIT: I see you already use AppVeyor so I guess it's something the tests don't cover :)

Believe me, without AppVeyor we would be totally lost on the Windows side. But I have not thought about this issue before (i.e. putting everything below src etc.), but a fix for this issue will include such a test.

bep on 8 Jun 2017

I always try to organize my folders and files in a strict way hence why I choose a subdirectory. It was the first thing I hit when I tried Hugo and being that it's the first time I play with Hugo I though it was my fault somewhere :/

XhmikosR on 8 Jun 2017

Same here, I first thought I've made a mistake. But when I checked out some other existing Hugo site from GitHub I've ran into the same issue. Gladfully, I've then found this issue here.

dominique-mueller on 8 Jun 2017

@bep: this shouldn't be tagged as an enhancement. It's a regression because it worked up until some point.

XhmikosR on 8 Jul 2017

@XhmikosR no, this has always behaved like this, and this works fine if you use "\\" slashes.

bep on 8 Jul 2017

Bit then it will fail on non Windows won't it?

On Jul 8, 2017 10:48, "Bjørn Erik Pedersen" notifications@github.com
wrote:

@XhmikosR https://github.com/xhmikosr no, this has always behaved like
this, and this works fine if you use "" slashes.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gohugoio/hugo/issues/3568#issuecomment-313841191, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAVVtWGfDA1HhzqcT_Zyc0IXgia6ahEDks5sLzQwgaJpZM4NzICW
.

XhmikosR on 8 Jul 2017

Bit then it will fail on non Windows won't it?

No, I'm pretty sure that will still work on *nix, but as I said, we will fix this, eventually.

bep on 8 Jul 2017

This error can be recreated on Linux (OpenSuSE 42.1) and MacOS Sierra (10.12.5) with hugo 0.25.1.

Three conditions must be met:

the --layoutDir argument is given on the command line
the argument to the --layoutDir argument contains a non-terminating slash
the directory pointed to by the --layoutDir is not empty

So if you fire up a new hugo directory, download a theme, config and run hugo, you won't have a problem. But as soon as you put something (even an empty file) in the layouts directory, it fails:

MacBook-Pro:/tmp/hugotest> uname -a
Darwin MacBook-Pro-3 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

MacBook-Pro:/tmp/hugotest> hugo version
Hugo Static Site Generator v0.25.1 darwin/amd64 BuildDate: 2017-07-10T03:57:59-03:00

MacBook-Pro:/tmp/hugotest> ls
archetypes      config.toml     content         data            layouts         public          static          themes

MacBook-Pro:/tmp/hugotest> ls layouts/
MacBook-Pro:/tmp/hugotest>

MacBook-Pro:/tmp/hugotest> hugo --quiet --themesDir ./themes/
MacBook-Pro:/tmp/hugotest>

MacBook-Pro:/tmp/hugotest> hugo --themesDir ./themes/ --layoutDir ./layouts --quiet
MacBook-Pro:/tmp/hugotest>

MacBook-Pro:/tmp/hugotest> touch layouts/index.html
MacBook-Pro:/tmp/hugotest> hugo --themesDir ./themes/ --layoutDir ./layouts --quiet                          
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/gohugoio/hugo/tpl/tplimpl.(*templateHandler).loadTemplates.func1(0xc42037e7b0, 0x2e, 0x1b5f600, 0xc4202e9c70, 0x0, 0x
0, 0xc420341370, 0x10879bb)
        /Users/bep/go/src/github.com/gohugoio/hugo/tpl/tplimpl/template.go:448 +0xd2e
github.com/gohugoio/hugo/vendor/github.com/spf13/afero.walk(0x1b61260, 0x1bcf900, 0xc42037e7b0, 0x2e, 0x1b5f600, 0xc4202e9c70, 0
xc42037e690, 0x0, 0x2e)
        /Users/bep/go/src/github.com/gohugoio/hugo/vendor/github.com/spf13/afero/path.go:44 +0x81
github.com/gohugoio/hugo/vendor/github.com/spf13/afero.Walk(0x1b61260, 0x1bcf900, 0xc42037e7b0, 0x2e, 0xc42037e690, 0x1, 0x1)
        /Users/bep/go/src/github.com/gohugoio/hugo/vendor/github.com/spf13/afero/path.go:107 +0x103
github.com/gohugoio/hugo/helpers.SymbolicWalk(0x1b61260, 0x1bcf900, 0xc42037e660, 0x23, 0xc42037e690, 0x2, 0x23)
        /Users/bep/go/src/github.com/gohugoio/hugo/helpers/path.go:513 +0x2ef
github.com/gohugoio/hugo/tpl/tplimpl.(*templateHandler).loadTemplates(0xc4202ea980, 0xc42037e660, 0x23, 0x0, 0x0)
        /Users/bep/go/src/github.com/gohugoio/hugo/tpl/tplimpl/template.go:480 +0x200
github.com/gohugoio/hugo/tpl/tplimpl.(*templateHandler).LoadTemplates(0xc4202ea980, 0xc42037e660, 0x23, 0x0, 0x0)
        /Users/bep/go/src/github.com/gohugoio/hugo/tpl/tplimpl/template.go:290 +0x53
github.com/gohugoio/hugo/hugolib.(*Site).withSiteTemplates.func1(0x1b5fba0, 0xc4202ea980, 0x0, 0x1bcfc18)
        /Users/bep/go/src/github.com/gohugoio/hugo/hugolib/hugo_sites.go:159 +0x80
github.com/gohugoio/hugo/tpl/tplimpl.(*TemplateProvider).Update(0x0, 0xc42027ad80, 0x0, 0x0)
        /Users/bep/go/src/github.com/gohugoio/hugo/tpl/tplimpl/templateProvider.go:35 +0x4c4
github.com/gohugoio/hugo/deps.(*Deps).LoadResources(0xc42027ad80, 0xc420182300, 0xc4202e6d80)
        /Users/bep/go/src/github.com/gohugoio/hugo/deps/deps.go:68 +0x72
github.com/gohugoio/hugo/hugolib.applyDepsIfNeeded(0xc420070100, 0xc420182300, 0xc4202e6d80, 0x1b5fd80, 0xc420165770, 0x1b587e0,
 0x0, 0xc4202cf200, 0x1b58760, 0xc4202ec260, ...)
        /Users/bep/go/src/github.com/gohugoio/hugo/hugolib/hugo_sites.go:130 +0x301
github.com/gohugoio/hugo/hugolib.newHugoSites(0xc420070100, 0xc420182300, 0x0, 0x1b5fd80, 0xc420165770, 0x0, 0x0, 0x0, 0x0, 0x0,
 ...)
        /Users/bep/go/src/github.com/gohugoio/hugo/hugolib/hugo_sites.go:89 +0x2e6
github.com/gohugoio/hugo/hugolib.NewHugoSites(0xc420070100, 0xc420182300, 0x0, 0x1b5fd80, 0xc420165770, 0x0, 0x0, 0x0, 0x0, 0x0,
 ...)
        /Users/bep/go/src/github.com/gohugoio/hugo/hugolib/hugo_sites.go:154 +0xce
github.com/gohugoio/hugo/commands.(*commandeer).initSites(0xc420254480, 0x17, 0xc420248300)
        /Users/bep/go/src/github.com/gohugoio/hugo/commands/hugo.go:733 +0x56
github.com/gohugoio/hugo/commands.(*commandeer).buildSites(0xc420254480, 0x0, 0x0, 0x10)
        /Users/bep/go/src/github.com/gohugoio/hugo/commands/hugo.go:744 +0x32
github.com/gohugoio/hugo/commands.(*commandeer).build(0xc420254480, 0x0, 0x0, 0x0, 0x0, 0x0)
        /Users/bep/go/src/github.com/gohugoio/hugo/commands/hugo.go:521 +0xa7
github.com/gohugoio/hugo/commands.glob..func8(0x1ba9aa0, 0xc420018af0, 0x0, 0x5, 0x0, 0x0)
        /Users/bep/go/src/github.com/gohugoio/hugo/commands/hugo.go:133 +0xb2
github.com/gohugoio/hugo/vendor/github.com/spf13/cobra.(*Command).execute(0x1ba9aa0, 0xc4200100d0, 0x5, 0x5, 0x1ba9aa0, 0xc42001
00d0)
        /Users/bep/go/src/github.com/gohugoio/hugo/vendor/github.com/spf13/cobra/command.go:647 +0x3f8
github.com/gohugoio/hugo/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x1ba9aa0, 0x1811fa8, 0x100610c, 0xc4200100b8)
        /Users/bep/go/src/github.com/gohugoio/hugo/vendor/github.com/spf13/cobra/command.go:726 +0x339
github.com/gohugoio/hugo/commands.Execute()
        /Users/bep/go/src/github.com/gohugoio/hugo/commands/hugo.go:172 +0x60
main.main()
        /Users/bep/go/src/github.com/gohugoio/hugo/main.go:27 +0x36

Interestingly, if you use a trailing slash on the layouts directory, you see a fencepost error in the path:

MacBook-Pro:/tmp/hugotest> hugo --themesDir ./themes/ --layoutDir layouts/ --quiet
ERROR 2017/07/13 00:10:28 Failed to resolve template in path "/tmp/hugotest/layouts/index.html": open /tmp/hugotest/layouts/ndex.html: no such file or directory

Also interesting, if you don't specify a layouts directory, everything works fine:

MacBook-Pro:/tmp/hugotest> hugo --themesDir ./themes/ --quiet
MacBook-Pro:/tmp/hugotest>

That a --layoutDir argument triggers it makes sense given that the stack trace points at line 448 of hugo/tpl/tplimpl/template.go which is pretty explicitly operating on the path of the layout directory:

template.go lines 446 - 448:

li := strings.LastIndex(path, layoutDir) + len(layoutDir) + 1
relPath := path[li:]
templateDir := path[:li-len(layoutDir)-1]

mumblecrunch on 13 Jul 2017

❤1

Something else I noticed when I used \\: Hugo can't find the root-relative assets in places like imageConfig:

{{ with (imageConfig (printf "/static/assets/img/foo/bar/%s" ($.Scratch.Get "imgFilename"))) }} width="{{ .Width }}"{{ end }}

C:\Users\xmr\Desktop\foobar>hugo --config=C:\Users\xmr\Desktop\foobar\config-common.yml,C:\Users\xmr\Desktop\foobar\config-development.yml
Started building sites ...
ERROR 2017/09/25 16:51:21 error processing shortcode "shortcodes/foo.html" for page "foo.md": template: shortcodes/foo.html:9:168: executing "shortcodes/foo.html" at <imageConfig (printf ...>:
error calling imageConfig: open C:\Users\xmr\Desktop\foobar\static\assets\img\foo\bar\foobar.png: The system cannot find the path specified.

XhmikosR on 25 Sep 2017

@XhmikosR the imageConfig and some other friends are restricted to the workingDir. You don't say where your static dir really is located, making it hard for me to guess.

A note to @mumblecrunch above: Your issue is related, but different, and the "fix" @kropp added for it breaks lots of other things. Could you pull that out into its own issue about the layoutDir?

bep on 28 Oct 2017

@XhmikosR I suspect, though, that your confusion is about what path is used for imageConfig. So if your Hugo project is at /my/project then imageConfig "/static/mylogo.png" will read from /my/project/static/mylogo.png. No more magic. This is also how readFile and readDir works.

bep on 28 Oct 2017

@bep: I didn't make any change in imageConfig's path so it's like I had it before switching to \\; /static/foo/etc. Shouldn't this take into consideration the staticdir config? Because currently it seems imageConfig is related to the dir the config is and not the staticdir value.

So basically my layout is:

<root>
  config.yml
    <src>
      content
      data
      layouts
      static

So, doing {{ with (imageConfig (printf "/static/assets/img/foo/bar/%s" ($.Scratch.Get "imgFilename"))) }} width="{{ .Width }}"{{ end }} I expect Hugo to take into consideration the staticdir value I set in config.yml. Judging by the error it seems imageConfig looks in root/static/.

XhmikosR on 29 Oct 2017

@XhmikosR Please use https://discourse.gohugo.io/ for questions/troubleshooting. Also see Hugo Documentation.

bep on 29 Oct 2017

A note to @mumblecrunch above: Your issue is related, but different, and the "fix" @kropp added for it breaks lots of other things. Could you pull that out into its own issue about the layoutDir?

Done. See issue #4024

mumblecrunch on 29 Oct 2017

Any news on this? I still have to maintain separate config files for Windows (my working machine) and Linux (the CI).

dominique-mueller on 29 Jun 2018

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

stale[bot] on 1 Jun 2019

I'm unable to reproduce this bug when testing with Hugo Static Site Generator v0.58.3-4AAC02D4 windows/amd64 BuildDate: 2019-09-19T15:29:19Z installed using Chocolatey.

sjansen on 14 Oct 2019

I'd need to try this again because it's been so long. Also, I never did actually used this folder structure due to this issue.

XhmikosR on 14 Oct 2019

I just tried this and now there's no error but the layouts aren't being picked up.

publishDir:             "../_site"
archetypeDir:           "archetypes"
assetDir:               "assets"
contentDir:             "content"
dataDir:                "data"
layoutDir:              "layouts"
staticDir:              "static"
themesDir:              "themes"

C:\Users\xmr\Desktop\foo>npm run docs-serve

> [email protected] docs-serve C:\Users\xmr\Desktop\foo
> hugo server --config docs/config.yml --port 4000 --disableFastRender

Building sites … WARN 2019/10/16 11:04:25 found no layout file for "HTML" for "home": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.

                   | EN
+------------------+----+
  Pages            |  1
  Paginator pages  |  0
  Non-page files   |  0
  Static files     |  0
  Processed images |  0
  Aliases          |  0
  Sitemaps         |  1
  Cleaned          |  0

Total in 5 ms
Watching for config changes in C:\Users\xmr\Desktop\foo\docs\config.yml
Environment: "development"
Serving pages from memory
Web Server is available at http://localhost:4000/ (bind address 127.0.0.1)
Press Ctrl+C to stop

It's probably not the same issue, though. But the result is the same, the site doesn't build.

XhmikosR on 16 Oct 2019

Was this page helpful?

0 / 5 - 0 ratings