hugo creating invalid timestamps in folder public

Created on 3 Aug 2019  ·  38Comments  ·  Source: gohugoio/hugo

I am using version 0.56.3 on MacOS 10.14.6 in rstudio.
There are problems with the folders css fonts images js and post in the folder public.
Here are is the listing from the theme folder static

drwxr-xr-x  4 neuwirth  staff    128 Jul 31 19:00 css
drwxr-xr-x  6 neuwirth  staff    192 Jul 31 19:00 fonts
drwxr-xr-x  5 neuwirth  staff    160 Jul 31 19:04 images
drwxr-xr-x  3 neuwirth  staff     96 Jul 31 19:00 js

And here ist the listing in public 

drwxr-xr-x  4 neuwirth  staff    128 Aug 30  1754 css
drwxr-xr-x  6 neuwirth  staff    192 Aug 30  1754 fonts
drwxr-xr-x  5 neuwirth  staff    160 Aug 30  1754 images
drwxr-xr-x  3 neuwirth  staff     96 Aug 30  1754 js
drwxr-xr-x  5 neuwirth  staff    160 Aug 30  1754 post

time is not a valid value in the public directory

Bug
Source
picture neuwirthe
👍8

Most helpful comment

I came across this problem a few days ago. I did a bit of digging and found where the problem occurs but sadly a full solution was not immediately obvious given my knowledge of the hugo source code. But here is what I found.
The 1754 date arises because the hugofs layer uses an empty time.Time structure for ModTime in dirNameOnlyFileInfo. Specifically hugofs/fileinfo.go

func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
    return time.Time{}
}

If I replace return time.Time() with return time.Now() then the folder gets the current date rather than one in the past.

It isn't obvious to me what the correct fix for that would be given that dirNameFileInfo only stores the leaf name so does not have the path to the original folder to read information from. Indeed whilst I did not unwrap the full layering of file systems that is used for this it may be that the correct fix may lie in one of the readOnly/union file system layers that hugo implements.

picture oldelvet on 14 Jan 2020
❤5

All 38 comments

I have encountered the same problem; v0.57.2 linux/amd64 BuildDate: 2019-08-18T09:32:21Z.
It's only a couple of directories and a file for me, but it's annoying because rsync kvetches about it. Looking at an example, it's basically the mtime which is wrong: 

Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: fd03h/64771d    Inode: 4470675     Links: 7
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    alex)   Gid: ( 1000/    alex)
Context: unconfined_u:object_r:user_home_t:s0
Access: 1754-08-30 22:42:26.128654848 -0001
Modify: 1754-08-30 22:42:26.128654848 -0001
Change: 2019-08-25 10:04:14.602861356 +0100
Birth: 2019-08-25 10:04:14.579861765 +0100

atime gets updated once I've run rsync obviously, but it still doesn't like the old mtime.

I've had a look at a few things and there's no obvious reason this is happening. I've stat'd the various files at the leaves of these directories, and they all look correct. But then the bad mtime seems to be propagating up the hierarchy. All these files are binary, so I would expect that it's only looking at the file metadata - maybe it's possible that some internal content is being picked up as a bad date, and that's then being treated as "newer" internally somehow?

alexhudson picture alexhudson on 25 Aug 2019
👍2

FWIW, the year 1754 appears to be a giveaway of some time.Time conversion being incorrectly defined, so there's likely some 0001-01-01 date or something being picked up somewhere, and it's not coming out as epoch because of an overflow:

https://codereview.appspot.com/5985059/diff/13001/src/pkg/time/time.go

Sadly I don't know the internal logic well enough to track that down, but the various logging/debug options don't appear to give any clues. Happy to try to step through the logic if I'm pointed in the right direction!

alexhudson picture alexhudson on 25 Aug 2019
👍1

I'm also seeing this problem. I find it happens if there's anything in the static directory.

Steps to reproduce

hugo new site quickstart
cd quickstart
touch static/test.txt
hugo
ls -l

output:

total 8
drwxr-xr-x  3 matt  staff   96  4 Sep 17:11 archetypes
-rw-r--r--  1 matt  staff   82  4 Sep 17:11 config.toml
drwxr-xr-x  2 matt  staff   64  4 Sep 17:11 content
drwxr-xr-x  2 matt  staff   64  4 Sep 17:11 data
drwxr-xr-x  2 matt  staff   64  4 Sep 17:11 layouts
drwxr-xr-x  7 matt  staff  224 30 Aug  1754 public
drwxr-xr-x  3 matt  staff   96  4 Sep 17:11 resources
drwxr-xr-x  3 matt  staff   96  4 Sep 17:12 static
drwxr-xr-x  2 matt  staff   64  4 Sep 17:11 themes

Note how I also get the same invalid timestamp on the public folder 30 Aug 1754.
This invalid timestamp is causing me problems further down in my build process. :(

Also happy to help debug if pointed in right direction, otherwise I'm going to have to write a _Post Build Cludge_.

mattburns picture mattburns on 4 Sep 2019

I'm also seeing this (Hugo v0.58.3 running on Ubuntu 18.04.3):

public, css, fonts, and images all show a timestamp of 30 Aug 1754.

Currently working around it by running touch public && touch public/ post-build.

jenh picture jenh on 17 Oct 2019

Yeah, I now have the same workaround in my gradle script...

// hack workaround while this bug exists: https://github.com/gohugoio/hugo/issues/6161
task fixHugoTimestamps(type: Exec, dependsOn: buildHugo) {
    commandLine 'find', 'path/to/public', '-type', 'd', '-exec', 'touch', '{}', '+' // touch all the directories
}
mattburns picture mattburns on 17 Oct 2019

i just ran into this today. i spent a few days writing a script to convert my old site to a more recent theme, got that working and was looking at the results and went, "What!??!!?!".

since i can recursively touch everything and then overwrite the static images with my original copy which should give me the correct times stamps i can work around this, but yuck.

flowerbug picture flowerbug on 7 Dec 2019

Following the steps outlined by @mattburns, I'm unable to reproduce this issue with the following Hugo releases:

Hugo Static Site Generator v0.61.0-DEV-0EFB00C2 linux/amd64 BuildDate: 2019-12-07T14:37:37-0600
Hugo Static Site Generator v0.58.3-4AAC02D4 linux/amd64 BuildDate: 2019-09-19T15:30:47Z
Hugo Static Site Generator v0.57.2-A849CB2D linux/amd64 BuildDate: 2019-08-17T17:54:13Z
Hugo Static Site Generator v0.56.3-F637A1EA linux/amd64 BuildDate: 2019-07-31T12:51:09Z

All tests in Archlinux.

$ uname -a
Linux arch 5.4.2-arch1-1 #1 SMP PREEMPT Thu, 05 Dec 2019 12:29:40 +0000 x86_64 GNU/Linux

Also tested v0.58.3 in Ubuntu 18.0.4.3 as mentioned by @jenh.

$ uname -a
Linux ubuntu 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
moorereason picture moorereason on 7 Dec 2019

Can others reporting this issue try the steps outlined by @mattburns?

moorereason picture moorereason on 7 Dec 2019

ok, i pulled the most recent version and built it locally. so i can say that this issue exists on an unmodified build (using my Debian testing machine):

$ hugo version
Hugo Static Site Generator v0.61.0-DEV linux/amd64 BuildDate: unknown

since the above patch is for something called gradle i can't do anything with it that makes sense to me (i've never looked at hugo or go before in any way other than using it).

anyone translate that into something i can plug into my local machine (and please say where since i don't have more clues at the moment). :)

flowerbug picture flowerbug on 8 Dec 2019

ok, so the default value for 32 bits is being used at least that is what i'm seeing in the results.

all my directories are set up with Dec 13 1901

from nosing around i see that:

strtotime 32 bit version would have this as the low part of the range.

The valid range of a timestamp is typically from Fri, 13 Dec 1901 20:45:54 UTC to Tue, 19 Jan 2038 03:14:07 UTC. (These are the dates that correspond to the minimum and maximum values for a 32-bit signed integer.)

~/production/public/img$ stat beans
File: beans
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 802h/2050d Inode: 395433 Links: 3
Access: (0700/drwx------) Uid: ( 1000/ me) Gid: ( 1000/ me)
Access: 2019-12-08 18:03:34.306753261 -0500
Modify: 1901-12-13 15:45:52.000000000 -0500
Change: 2019-12-08 17:59:04.217171546 -0500
Birth: -
$ stat ../../static/img/beans
File: ../../static/img/beans
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 802h/2050d Inode: 394334 Links: 3
Access: (0700/drwx------) Uid: ( 1000/ me) Gid: ( 1000/ me)
Access: 2019-12-08 17:04:09.165742766 -0500
Modify: 2019-04-15 13:53:40.845871032 -0400
Change: 2019-12-07 11:59:59.466040113 -0500
Birth: -

so for some reason the original modification time is not being copied.

flowerbug picture flowerbug on 9 Dec 2019

@flowerbug,
I was referring to these steps from @mattburns:

hugo new site quickstart
cd quickstart
touch static/test.txt
hugo
ls -l
moorereason picture moorereason on 9 Dec 2019

@flowerbug,
I was referring to these steps from @mattburns:

hugo new site quickstart
cd quickstart
touch static/test.txt
hugo
ls -l

```
as you can see it only happens with subdirectories of static.

me@ant(112)~$ mkdir fred
me@ant(113)~$ cd fred
me@ant(114)~/fred$ hugo new site quickstart
Congratulations! Your new Hugo site is created in /home/me/fred/quickstart.

Just a few more steps and you're ready to go:

  1. Download a theme into the same-named folder.
    Choose a theme from https://themes.gohugo.io/ or
    create your own with the "hugo new theme " command.
  2. Perhaps you want to add some content. You can add single files
    with "hugo new /.".
  3. Start the built-in live server via "hugo server".

Visit https://gohugo.io/ for quickstart guide and full documentation.
me@ant(115)~/fred$ cd quickstart/
me@ant(116)~/fred/quickstart$ ls
total 28
drwx------ 2 me me 4096 Dec 8 18:41 archetypes
-rw------- 1 me me 82 Dec 8 18:41 config.toml
drwx------ 2 me me 4096 Dec 8 18:41 content
drwx------ 2 me me 4096 Dec 8 18:41 data
drwx------ 2 me me 4096 Dec 8 18:41 layouts
drwx------ 2 me me 4096 Dec 8 18:41 static
drwx------ 2 me me 4096 Dec 8 18:41 themes
me@ant(117)~/fred/quickstart$ touch static/test.txt
me@ant(118)~/fred/quickstart$ ls
total 28
drwx------ 2 me me 4096 Dec 8 18:41 archetypes
-rw------- 1 me me 82 Dec 8 18:41 config.toml
drwx------ 2 me me 4096 Dec 8 18:41 content
drwx------ 2 me me 4096 Dec 8 18:41 data
drwx------ 2 me me 4096 Dec 8 18:41 layouts
drwx------ 2 me me 4096 Dec 8 18:41 static
drwx------ 2 me me 4096 Dec 8 18:41 themes
me@ant(119)~/fred/quickstart$ hugo
Building sites … WARN 2019/12/08 18:42:02 found no layout file for "HTML" for "home": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
WARN 2019/12/08 18:42:02 found no layout file for "HTML" for "taxonomyTerm": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
WARN 2019/12/08 18:42:02 found no layout file for "HTML" for "taxonomyTerm": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.

               | EN

+------------------+----+
Pages | 3
Paginator pages | 0
Non-page files | 0
Static files | 1
Processed images | 0
Aliases | 0
Sitemaps | 1
Cleaned | 0

Total in 9 ms
me@ant(120)~/fred/quickstart$ ls -l
total 36
drwx------ 2 me me 4096 Dec 8 18:41 archetypes
-rw------- 1 me me 82 Dec 8 18:41 config.toml
drwx------ 2 me me 4096 Dec 8 18:41 content
drwx------ 2 me me 4096 Dec 8 18:41 data
drwx------ 2 me me 4096 Dec 8 18:41 layouts
drwx------ 4 me me 4096 Dec 8 18:42 public
drwx------ 3 me me 4096 Dec 8 18:42 resources
drwx------ 2 me me 4096 Dec 8 18:41 static
drwx------ 2 me me 4096 Dec 8 18:41 themes
me@ant(121)~/fred/quickstart$ ls static
total 0
-rw------- 1 me me 0 Dec 8 18:41 test.txt
me@ant(122)~/fred/quickstart$ which hugo
/home/me/go/bin/hugo
me@ant(123)~/fred/quickstart$ ls public
total 16
drwx------ 2 me me 4096 Dec 8 18:42 categories
-rw------- 1 me me 467 Dec 8 18:42 index.xml
-rw------- 1 me me 355 Dec 8 18:42 sitemap.xml
drwx------ 2 me me 4096 Dec 8 18:42 tags
-rw------- 1 me me 0 Dec 8 18:41 test.txt
me@ant(124)~/fred/quickstart$ cd static
me@ant(125)~/fred/quickstart/static$ ls
total 0
-rw------- 1 me me 0 Dec 8 18:41 test.txt
me@ant(126)~/fred/quickstart/static$ mkdir subdir
me@ant(127)~/fred/quickstart/static$ cd subdir
me@ant(128)~/fred/quickstart/static/subdir$ touch othertest.txt
me@ant(129)~/fred/quickstart/static/subdir$ ls
total 0
-rw------- 1 me me 0 Dec 8 18:43 othertest.txt
me@ant(130)~/fred/quickstart/static/subdir$ cd ..
me@ant(131)~/fred/quickstart/static$ cd ..
me@ant(132)~/fred/quickstart$ hugo
Building sites … WARN 2019/12/08 18:43:52 found no layout file for "HTML" for "taxonomyTerm": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
WARN 2019/12/08 18:43:52 found no layout file for "HTML" for "home": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
WARN 2019/12/08 18:43:52 found no layout file for "HTML" for "taxonomyTerm": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.

               | EN

+------------------+----+
Pages | 3
Paginator pages | 0
Non-page files | 0
Static files | 2
Processed images | 0
Aliases | 0
Sitemaps | 1
Cleaned | 0

Total in 8 ms
me@ant(133)~/fred/quickstart$ ls -l public/img
ls: cannot access 'public/img': No such file or directory
me@ant(134)~/fred/quickstart$ ls -l public/subdir
total 0
-rw------- 1 me me 0 Dec 8 18:43 othertest.txt
me@ant(135)~/fred/quickstart$ ls -l public/
total 20
drwx------ 2 me me 4096 Dec 8 18:42 categories
-rw------- 1 me me 467 Dec 8 18:43 index.xml
-rw------- 1 me me 355 Dec 8 18:43 sitemap.xml
drwx------ 2 me me 4096 Dec 13 1901 subdir
drwx------ 2 me me 4096 Dec 8 18:42 tags
-rw------- 1 me me 0 Dec 8 18:41 test.txt
me@ant(136)~/fred/quickstart$ ls public/subdir
total 0
-rw------- 1 me me 0 Dec 8 18:43 othertest.txt
me@ant(137)~/fred/quickstart$ cd public
me@ant(138)~/fred/quickstart/public$ stat subdir/
File: subdir/
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 802h/2050d Inode: 659566 Links: 2
Access: (0700/drwx------) Uid: ( 1000/ me) Gid: ( 1000/ me)
Access: 2019-12-08 18:44:17.344143773 -0500
Modify: 1901-12-13 15:45:52.000000000 -0500
Change: 2019-12-08 18:43:52.411830451 -0500
Birth: -
me@ant(139)~/fred/quickstart/public$ ls
total 20
drwx------ 2 me me 4096 Dec 8 18:42 categories
-rw------- 1 me me 467 Dec 8 18:43 index.xml
-rw------- 1 me me 355 Dec 8 18:43 sitemap.xml
drwx------ 2 me me 4096 Dec 13 1901 subdir
drwx------ 2 me me 4096 Dec 8 18:42 tags
-rw------- 1 me me 0 Dec 8 18:41 test.txt
me@ant(140)~/fred/quickstart/public$ cd subdir
me@ant(141)~/fred/quickstart/public/subdir$ ls
total 0
-rw------- 1 me me 0 Dec 8 18:43 othertest.txt
me@ant(142)~/fred/quickstart/public/subdir$ stat othertest.txt
File: othertest.txt
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 802h/2050d Inode: 659567 Links: 1
Access: (0600/-rw-------) Uid: ( 1000/ me) Gid: ( 1000/ me)
Access: 2019-12-08 18:43:46.731759305 -0500
Modify: 2019-12-08 18:43:46.731759305 -0500
Change: 2019-12-08 18:43:52.411830451 -0500
Birth: -
me@ant(143)~/fred/quickstart/public/subdir$
```

public itself is incorrect too, but only on the second run. the first run it is correct.

me@ant(32)~/fred/quickstart$ ls
total 36
drwx------ 2 me me 4096 Dec  8 18:41 archetypes
-rw------- 1 me me   82 Dec  8 18:41 config.toml
drwx------ 2 me me 4096 Dec  8 18:41 content
drwx------ 2 me me 4096 Dec  8 18:41 data
drwx------ 2 me me 4096 Dec  8 18:41 layouts
drwx------ 5 me me 4096 Dec 13  1901 public
drwx------ 3 me me 4096 Dec  8 18:42 resources
drwx------ 3 me me 4096 Dec  8 18:43 static
drwx------ 2 me me 4096 Dec  8 18:41 themes
flowerbug picture flowerbug on 9 Dec 2019

Thank you, @flowerbug! I'm able to reproduce with master by simply running Hugo twice.

$ hugo new site quickstart >/dev/null
$ cd quickstart
$ hugo >/dev/null
$ ls -ld public/
drwxr-xr-x 1 me users 68 Dec  8 18:57 public/
$ hugo >/dev/null
$ ls -ld public/
drwxr-xr-x 1 me users 68 Aug 30  1754 public/
moorereason picture moorereason on 9 Dec 2019

you're welcome. :) i was going to look at the code itself, but i've never even looked at go before. better use of my time at the moment to just provide as much information as i can. if i can get some time tomorrow i'll dig into it and see if i can figure it out. don't expect miracles though...

flowerbug picture flowerbug on 9 Dec 2019

I am doing this on a Mac with the homebrew version of hugo.
The issue is that this way, my posts do not have access to the css files (in css and in rmarkdown-lib) and display incorrectly.
Is there a workaround?
just doing
find . -type d -exec touch {} +
in public does not seem to solve the problem, because after the next build, the old dates return.

neuwirthe picture neuwirthe on 11 Dec 2019

I am doing this on a Mac with the homebrew version of hugo.
The issue is that this way, my posts do not have access to the css files (in css and in rmarkdown-lib) and display incorrectly.
Is there a workaround?
just doing
find . -type d -exec touch {} +
in public does not seem to solve the problem, because after the next build, the old dates return.

yes, you have to do that each time... for a really big site with a lot of pictures this makes it hard to update if your sync program isn't sophisticated enough to know that the files haven't all been updated. what i do to get around that is do the touch first and then copy my static img directory over the one that got touched so all the time stamps are corrected.

i did try to look at the code but i didn't make much progress (i'm not familiar with go at all). i did get delve installed and could use it, but i just don't know the syntax of the language or debugger well enough to set the breakpoints i needed to set to watch that value to see what was changing the value.

flowerbug picture flowerbug on 12 Dec 2019

I encountered the same issue on Windows 10, with release hugo_0.61.0_Windows-64bit.

image

jcyrss picture jcyrss on 17 Dec 2019

what is the latest version that worked?

unfortunately the hugo theme i'm using requires a more recent version so this bug has to be worked around each time i'm making any change to my website.

flowerbug picture flowerbug on 14 Jan 2020

I came across this problem a few days ago. I did a bit of digging and found where the problem occurs but sadly a full solution was not immediately obvious given my knowledge of the hugo source code. But here is what I found.
The 1754 date arises because the hugofs layer uses an empty time.Time structure for ModTime in dirNameOnlyFileInfo. Specifically hugofs/fileinfo.go

func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
    return time.Time{}
}

If I replace return time.Time() with return time.Now() then the folder gets the current date rather than one in the past.

It isn't obvious to me what the correct fix for that would be given that dirNameFileInfo only stores the leaf name so does not have the path to the original folder to read information from. Indeed whilst I did not unwrap the full layering of file systems that is used for this it may be that the correct fix may lie in one of the readOnly/union file system layers that hugo implements.

oldelvet picture oldelvet on 14 Jan 2020
❤5

the correct fix is to use the date from the item being copied. is there some place where only the name is being copied when in fact the real intent was to copy the name and all the other file or directory meta data?

a git bisect should allow the bug introduction to be found.

note that i do not want the current date to be used because that would mean that for that item it would be uploaded by a file handling utility which uses the combination of Source File Date Newer or File Size Different as an indicator of which files that actually need to be uploaded. when you have a lot of pictures on a website you don't want to re-upload them each time.

flowerbug picture flowerbug on 14 Jan 2020

I can confirm that the issue occurs also in version: Hugo Static Site Generator v0.62.2-83E50184 linux/amd64 BuildDate: 2020-01-05T18:51:38Z

Enrico204 picture Enrico204 on 22 Jan 2020

My Hugo which installed from Homebrew has the same problem: hugo Static Site Generator v0.62.0/extended darwin/amd64 BuildDate: unknown.

Which resulted in https://github.com/ogham/exa panic:

ringsaturn picture ringsaturn on 29 Jan 2020
👍2

I stumbled upon it due to rsync not being able to sync it:

Time value of /var/.../hugo/public truncated on receiver.
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]

With v0.64.1. Yesterday's date was 2299-01-27, today's is 1754-08-30. Please also note the weird timezone:

ls yesterday: drwxr-xr-x 36 root root 4096 Jan 27  2299 public
ls today: drwxr-xr-x 36 root root 4096 Aug 30  1754 public
stat today: Modify: 1754-08-30 23:37:09.128654848 +0053
Jonny007-MKD picture Jonny007-MKD on 16 Feb 2020

[edit, nope, the bug is still there after all]

version:

Hugo Static Site Generator v0.65.0-DEV linux/amd64 BuildDate: unknown

yesterday i had to do some updates and the bug is still there after all... i just have to remind myself to delete the public directory where output goes before running hugo and it seems to work. if the output directory already exists then it doesn't work.

flowerbug picture flowerbug on 16 Feb 2020
👍1 
> hugo version
Hugo Static Site Generator v0.65.3/extended darwin/amd64 BuildDate: unknown

Still have this bug, which causes exa (ls replacement) to crash.

On Mac there is standard command line utility SetFile that can correct Creation date.
So we can use SetFile -d "01/01/2020 00:00:00" public for instance to correct the date.
Yet to figure it out how to automate this until the bug is fixed.

You can check attributes with GetFileInfo public

denrasulev picture denrasulev on 26 Feb 2020

The previous version (63?) was still generating bad dates when I tested before upgrading hugo.

Upgraded to latest version via Homebrew today:

Hugo Static Site Generator v0.65.3/extended darwin/amd64 BuildDate: unknown

Dates now seem to be correct for me. No longer crashes when using exa.

👍

wlbentley picture wlbentley on 26 Feb 2020

My hugo version: Hugo Static Site Generator v0.65.3-211BA42A/extended linux/amd64 BuildDate: 2020-02-23T10:07:00Z

When running hugo without having a public/ folder, the timestamp is correct:

➜ rm -rf public 
➜ hugo > /dev/null
➜ ls -la | grep "public"
drwxr-xr-x.  7 eirik eirik 4096 Feb 29 22:24 public

However, if the public/ folder is present, the timestamp gets mangled:

➜ ls -la | grep "public"
drwxr-xr-x.  7 eirik eirik 4096 Feb 29 22:24 public
➜ hugo > /dev/null
➜ ls -la | grep "public"
drwxr-xr-x.  7 eirik eirik 4096 Dec 13  1901 public

Make is reporting the following:

make: public/: Timestamp out of range; substituting 2514-05-30 03:53:03.999999999
make: Warning: File 'public/' has modification time 15596857564 s in the future
R0flcopt3r picture R0flcopt3r on 29 Feb 2020 
Hugo Static Site Generator v0.65.3/extended darwin/amd64 BuildDate: unknown

Dates now seem to be correct for me. No longer crashes when using exa.

👍

Oops, I guess that was a little premature. Hugo is still generating date of Aug 30 1754 for the public/ directory itself. The generated files all seem to have the correct date now.

I deleted the entire public/ folder before running hugo, and the newly generated directory has the bad date.

wlbentley picture wlbentley on 4 Mar 2020

I can confirm this, that the public directory gets date Aug 30 1754 with the homebrew version of hugo in MacOS 10.14.6
But this is very strange:
ls:
drwxr-xr-x 23 neuwirth staff 736 Aug 30 1754 public
stat -x public :
File: "public"
Size: 736 FileType: Directory
Mode: (0755/drwxr-xr-x) Uid: ( 501/neuwirth) Gid: ( 20/ staff)
Device: 1,8 Inode: 12957611732 Links: 23
Access: Sat Mar 7 11:35:52 2020
Modify: Fri Aug 30 23:49:03 1754
Change: Thu Mar 5 00:05:49 2020

neuwirthe picture neuwirthe on 7 Mar 2020

This bug actually prevents hugo from working on illumos, where I suppose we are somewhat more stringent with the timestamp values we accept in the file system:

$ hugo
Total in 70 ms
Error: Error copying static files: chtimes /var/tmp/BLAG/public/: value too large for defined data type

Digging in, the call that's failing for us is utimensat(2):

$ truss -f -v utimensat -o /tmp/sigh.out /ws/hugo/thrice/hugo/hugo
Total in 129 ms
Error: Error copying static files: chtimes /var/tmp/BLAG/public/: value too large for defined data type

$ grep -A2 OVERFLOW /tmp/sigh.out 
61343/9:    utimensat(AT_FDCWD, "/var/tmp/BLAG/public/", 0xC0008AB810, 0) Err#79 EOVERFLOW
61343/9:        at = Aug 30 22:43:41 UTC 1754  [ 18446744066914187037.128654848 ]
61343/9:        mt = Aug 30 22:43:41 UTC 1754  [ 18446744066914187037.128654848 ]

Using DTrace I was able to get a more complete stack for that call:

$ pfexec dtrace -w -n 'syscall::utimesys:entry /arg0 == 1/ { stop(); ustack(); }'
CPU     ID                    FUNCTION:NAME
  2   8105                   utimesys:entry
   libc.so.1`_syscall6+0x1b
   hugo`runtime.asmsysvicall6+0x5a
   hugo`syscall.utimensat+0xeb
   hugo`os.Chtimes+0x182
   hugo`github.com/spf13/afero.(*OsFs).Chtimes+0x9d
   hugo`github.com/spf13/fsync.(*Syncer).syncstats+0x305
   hugo`github.com/spf13/fsync.(*Syncer).sync+0x53f
   hugo`github.com/spf13/fsync.(*Syncer).syncRecover+0x87
   hugo`github.com/spf13/fsync.(*Syncer).Sync+0x110
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStaticTo+0x2fe
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStaticTo-fm+0x34
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).doWithPublishDirs+0x183
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStatic+0x52
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).fullBuild.func2+0x37
   hugo`golang.org/x/sync/errgroup.(*Group).Go.func1+0x59
   hugo`runtime.goexit+0x1

That time value looked suspicious so I spent rather a long time chasing down where it comes from, and it seems in the end to be from code in Hugo itself -- albeit called through the inpenetrable web of indirection that is the afero library.

RootMappingFs.doLstat() will, under some conditions, knock together a fake os.FileInfo using newDirNameOnlyFileInfo(). That routine constructs an object, dirNameOnlyFileInfo, which will return an invalid time.Time for its ModTime() implementation:

func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
        return time.Time{}
}

There would appear to be at least two simple choices for fixing this:

  • return time.Now(), which will update the timestamp on the build directory and which seems useful
  • return time.Unix(0, 0), which is perhaps closer to what may have potentially been intended here: a constant but obviously pretend timestamp

At any rate, this small diff appears to fix the issue for me:

diff --git a/hugofs/fileinfo.go b/hugofs/fileinfo.go
index 79d89a88..0a3ccd36 100644
--- a/hugofs/fileinfo.go
+++ b/hugofs/fileinfo.go
@@ -275,7 +275,7 @@ func (fi *dirNameOnlyFileInfo) Mode() os.FileMode {
 }

 func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
-       return time.Time{}
+       return time.Now()
 }

 func (fi *dirNameOnlyFileInfo) IsDir() bool {

I had a quick look, and there are a handful of other places that end up using time.Time{}. This appears somewhat dubious in general, and if the intent is for a common epoch timestamp, perhaps time.Unix(0, 0) may be better there -- at least if the date may end up in file system metadata.

jclulow picture jclulow on 9 Mar 2020
🎉4

LGTM - I don't have much time to look after these methods actually, but I was wondering: what is the reason behind this "mocked" version? Because probably here the error is in the hugo work flow (just wondering) - in other words, I can't see any valid reason to use a value initialized like that. Also, given the fact that these functions seems to be used in file creation or update, time.Now() might be a good solution (and maybe what was intended).

BTW, @jclulow this is a wonderful debug, thanks! :-D

Enrico204 picture Enrico204 on 11 Mar 2020

since the behavior keeps changing for me i don't know what to think of this any more. :)

right now with version:

Hugo Static Site Generator v0.66.0 linux/amd64 BuildDate: unknown

the only directory which is not having the correct date is the top level public directory:

drwx------ 14 me me 4096 Dec 13 1901 public

the subdirectories within public are now correctly copied and have the right date on them.

flowerbug picture flowerbug on 12 Mar 2020 
> hugo version
Hugo Static Site Generator v0.68.3/extended darwin/amd64 BuildDate: unknown

Problem still exists in the latest version. My solution until it's fixed is to enhance deploy script:

> cat ./deploy
hugo --minify --gc && rsync -az --delete public/ USER@HOST:/var/www/your.site.com/html/
SetFile -d $(date +%d/%m/%Y) public/
SetFile -m $(date +%d/%m/%Y) public/

This fixes the date for public folder in the directory.

denrasulev picture denrasulev on 29 Mar 2020
👍1

I've encounted this bug in latest hugo 0.69, and it causes the python 2.7 tar library (tarfile.py) to crash:

https://github.com/YunoHost/issues/issues/1570

farvardin picture farvardin on 22 Apr 2020

drwxr-xr-x 3 brooqs brooqs 4096 May 10 23:06 layouts
drwxr-xr-x 20 brooqs brooqs 4096 Dec 13 1901 public
drwxr-xr-x 20 brooqs brooqs 4096 Dec 13 1901 public1
-rw-r--r-- 1 brooqs brooqs 7479663 May 10 23:52 public.tar.gz
drwxr-xr-x 3 brooqs brooqs 4096 May 10 23:09 resources
drwxr-xr-x 5 brooqs brooqs 4096 May 10 23:06 static
drwxr-xr-x 3 brooqs brooqs 4096 May 10 23:06 themes
[brooqs@cybertron site]$ hugo version
Hugo Static Site Generator v0.70.0/extended linux/amd64 BuildDate: unknown
[brooqs@cybertron site]$

This bug still exist in 0.70.0

dhyaman picture dhyaman on 10 May 2020

This issue still exists in v0.71.1.

hugo version
Hugo Static Site Generator v0.71.1/extended darwin/amd64 BuildDate: unknown

drwxr-xr-x  62 staff   1.9K Aug 30  1754 public/
mrichman picture mrichman on 30 May 2020
👍1

I hit this bug too, after installing and testing exa. Not critical for me; just annoying. :)

RickCogley picture RickCogley on 19 Jul 2020

I just stumbled upon this issue while Googling for the particular timestamp. It appears it has to do with overflow caused by the Unix nanoseconds of the timestamp:

emptyTime := time.Time{}
println(emptyTime.IsZero())                          // true
println(emptyTime.UnixNano())                        // -6795364578871345152
println(time.Unix(0, emptyTime.UnixNano()).IsZero()) // false
println(time.Unix(0, emptyTime.UnixNano()).String()) // 1754-08-30 22:43:41.128654848 +0000 LMT

In other words, when you 'convert' an zero timestamp to unix nanoseconds then convert it back to time.Time the result isn't what you expected. I guess the solution is to manually handle converting to 0 and back when IsZero() == true. Although that might be a bit hacky since Unix epoch 0 is 01-01-1970. time.Unix(0, 0) != time.Time{}

Anyways, I hope this helps solving this issue.

reinkrul picture reinkrul on 12 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bep picture bep  ·  3Comments
vielmetti picture vielmetti  ·  3Comments
moorereason picture moorereason  ·  3Comments
kaushalmodi picture kaushalmodi  ·  3Comments
MunifTanjim picture MunifTanjim  ·  3Comments