Go: time: LoadLocation doesn't work on windows if Go is not installed

[iman75]

What version of Go are you using (go version)?

go version go1.8.3 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:UsersIman Akabigo
set GORACE=
set GOROOT=C:Go
set GOTOOLDIR=C:Gopkgtoolwindows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set PKG_CONFIG=pkg-config
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2

What did you do?

https://play.golang.org/p/VuJ3ofkdk8
package main

import (
"fmt"
"time"
"log"
)

func main() {
var ParisLocation, err = time.LoadLocation("Europe/Paris")
if(err != nil) {
log.Fatal(err)
}

fmt.Println(ParisLocation)

}

I built this basic go code and generated an executable file.
I have an error when I run this executable on a windows machine without Go installed because time.LoadLocation needs to aceess to "zoneinfo.zip" located at $GOROOT/lib/time/zoneinfo.zip.

What did you expect to see?

Europe/Paris

What did you see instead?

open c:golibtimezoneinfo.zip: The system cannot find the file specified.

[ianlancetaylor]

I don't see how we can fix this, at least not without bloating the size of almost every Go binary on Windows. I'm open to suggestions.

[skirmish]

On windows couldn't it (default to|try) the (current working directory|executable location) to load the zoneinfo.zip? That way you only can include the file when it is required. Yeah, it's an extra hoop to jump through for us windows users, but I guess it's not too big of an ordeal seeing as the executable has to be distributed somehow, one extra file might not be too bad...

[alexbrainman]

On windows couldn't it (default to|try) the (current working directory|executable location) to load the zoneinfo.zip?

Sounds simple enough for me.

Alex

[glycerine]

zoneinfo.zip is a mere 366776 bytes, a small number amongst friends. I think of a typical go binary as a 10MB +/- 5MB affair, and I would happily increase that by 366KB (3% of the mean, but small compared to the standard deviation) to have my binaries be portable to windows.

[leighmcculloch]

I don't see how we can fix this, at least not without bloating the size of almost every Go binary on Windows.

Providing the tzdata ourselves in the binary if a specific package is imported (e.g. time/zone) seems less worse to me than requiring every Windows user to install Go, which is much larger to download and is inconsistent with the static binary.

Could we duplicate the time.LoadLocation function to another package which also contains the tzdata, e.g. zone.LoadLocation. This new function would operate the same as time.LoadLocation, but would load from a tzdata file built into a new zone package. This would only introduce bloat to binaries that imported the zone package, and applications could choose to use time.LoadLocation or zone.LoadLocation. We could mark time.LoadLocation as deprecated, or at least mention in a comment on it about it's limitations and about the alternative. Having the basic time code and the zone code in separate packages would mean the bloat is only introduced if applications specifically need it.

This would probably also resolve issues #20629 and #20969.

[alexbrainman]

Just an idea. On windows we use a particular Windows registry key to determine our timezone, and then we use a map that maps that string into std / dst values that are ultimately used to look in the tzdata blob. The "registry key" to "std / dst" map is hard coded in "abbrs" map in time package. This map contains 133 values at this moment. I wonder if we could strip tzdata blob to leave only the data we actually use on windows. I don't have time to try and implement this now, perhaps others do.

Alex

[nathan-osman]

It would be awesome if there was a function that we could call to pass the contents of zoneinfo.zip to the time package. That way, the caller is responsible for determining where they want to put zoneinfo.zip and they have the option of embedding its contents in the executable.

[bradfitz]

@nathan-osman, Go 1.10 has https://beta.golang.org/pkg/time/#LoadLocationFromTZData (that was #20629)

[nathan-osman]

@bradfitz excellent, thanks! That will work perfectly.

[shabbyrobe]

I was pretty astonished by this. I can trivially build and deploy binaries to Windows just like I can everywhere else, but as soon as I try to work with a timezone, my application blows up. Go's tooling lulls me into a false sense of ease upon which I grow to rely, but it's a false promise. I'd much rather a larger binary than an... ahem... time-bomb like this ticking away.

[glycerine]

All, what's the best current workaround for this? I need to distribute a windows binary, that uses time, and I'd like it not to blow up on user's boxes.

import "time/zone" would be great, eventually. But what's the solution today?

Thank you!

[bradfitz]

The solution is to ship the tzdata yourself (linked into your binary or elsewhere) and then call https://golang.org/pkg/time/#LoadLocationFromTZData.

Or set the %ZONEINFO% environment variable to point to your shipped zoneinfo.zip.

[glycerine]

Thanks Brad.

For others, the data that is passed to LoadLocationFromTZData is a file extracted from inside zoneinfo.zip, such as the file America/New_York, not zoneinfo.zip itself.

[shabbyrobe]

This is the quickie solution I used yesterday to get moving again: https://gist.github.com/shabbyrobe/b9a7a56e25bd8e441b7b3fe39dbb04fa

It relies on something not unlike the now-infamous https://github.com/jteeuwen/go-bindata to generate a go file with that tzfiles map referenced in the gist, which contains the base64-encoded zoneinfo.zip file I grabbed from the GOROOT of another Windows machine that was allowed to have Go installed on it (not always an option).

The code was dashed out under the duress of extreme time pressure so it's not exactly nice (and comes with absolutely no warranty whatsoever) but hopefully it can help someone else get out of the same jam.

[leighmcculloch]

I created 4d63.com/tz because I find it more convenient to import a package than manually including a file to ride alongside an executable. It embeds Go's zoneinfo.zip and exports tz.LoadLocation(name) to load locations from the embedded tzdata.

[glycerine]

Awesome @leighmcculloch. That should really become part of the standard library (or rather just be the implementation of time.LoadLocation on Windows), but while waiting for that, its just great to have it available. In the meantime, all calls to time.LoadLocation, in order to not crash on windows/OS without zoneinfo, need to become calls to tz.LoadLocation() after
import "4d63.com/tz".

[glycerine]

I just spent 4 days debugging why my DLL would load on one windows host and crash on another. Time I won't ever get back. Yep, it was due to missing the %GOROOT%libtimezoneinfo.zip file on the deploy host. Pretty please include zoneinfo.zip with the Go runtime on windows. Windows DLLs can't be distributed to non-developers as it stands in go1.12.5.

[skirmish]

@glycerine Sorry you spent so much time on that, agreed it's not the best solution, but in reading the discussions related to time zones on windows, adding useless static data to every go program on windows (especially when it's not required) isn't a great solution either. Windows being the oddity because it does time zones differently. You're really left with one of the above solutions, distribute the zoneinfo.zip file with your app and explicitly load it, or set the %ZONEINFO% environment variable to point to your shipped zoneinfo.zip. I had similar issues nearly 2 years ago, but now, I know the fix and to me it's just a windows(ism) I remember to always have to do. Also it came up more recently when I was building scratch containers for Linux too along with a few other dependencies under the hood. I guess also using https://4d63.com/tz work too, possibly a much cleaner solution.

[shabbyrobe]

adding useless static data to every go program on windows (especially when it's not required) isn't a great solution either

It's less worse though. My experience was very similar to @glycerine.

This is one that could even sneak past a well-constructed build/test pipeline - you can do everything right and still be exposed to this gotcha simply by not realising your deployment target doesn't have the Go toolchain installed.

[glycerine]

adding useless static data to every go program on windows (especially when it's not required) isn't a great solution either.

I must vehemently disagree. This is the preferred solution. My Go binaries are 50MB.
My gosh. The timezone data is a tiny file, less than 400KB. Just add it. Good grief. Every program of any complexity will import the time package, and want to not crash on windows.

Sample fix, needs the niceties of review and convention applied, but for those of us who needed a solution yesterday (merges 4d63.com/tz into the time package):

https://github.com/glycerine/go/tree/timezone_windows_fix

[ianlancetaylor]

The timezone data file is nearly 800K now, due to #30099. Not that that really affects this calculation.

[leighmcculloch]

There are a couple competing things for me about this issue:

An attribute of Go binaries is that they are static, containing all their Go dependencies, so tzinfo not being included is a surprise. And, in the Go community using tools like packr, embedfiles, it is common to pack files into binaries rather than ship them sidecar, giving the sense people prefer things baked in.

But, TZ data is more like certificate authorities than application data, and we don't bundle certificate authorities into Go binaries because they are typically managed by sysadmins. Sysadmins probably care about TZ data less though making the utility of them being updatable/changeable less useful?

Since Go only pulls in packages if they're imported could we have tzinfo in a separate package that is only pulled in if it was required? Would it be possible to separate it from the time package? I'm guessing no.

[jfritz]

Add me to the list of people inconvenienced by this; philosophically my vote is for baking tz data into the binary given that the binaries are already relatively large. Whatever the fix is, I am surprised this issue was reported in 2017 and still remains for such a seemingly big issue (affecting tz use on entire platform).

[ianlancetaylor]

There are various ways to address this that are described above.

The question is: what should the default behavior be? I agree that the current default is a poor choice. But it's not obvious to me that growing the size of essentially every Windows Go binary by 800K is a better choice. So perhaps we need a way to make the default be to make the binary larger but have some way to disable that. But I don't know what the disabling mechanism would be.

[glycerine]

So perhaps we need a way to make the default be to make the binary larger but have some way to disable that. But I don't know what the disabling mechanism would be.

The disabling mechanism could be importing a "thintime" package instead of "time".

It might be worth taking the time to redo the API for thintime.Location in this suggested thintime package. If my suspicions are correct, the thing that really bloats the use of time.Location is that it uses strings as keys to pick a location. Since the compiler cannot know which strings will be used, it has to include all time locations. If it used something that the compiler could do dead-code elimination on (or the equivalent for data), then my program wouldn't need all 800KB of timezones. After all, I use but three locations at the moment, and would be happy to recompile if I needed others.

[mappu]

The alternative would be to reimplement large parts of this package, to use Windows time APIs directly instead of parsing a bundled tzdata.

I'm not sure if it's 100% feasible, but, clearly these APIs do exist:

• .NET has a copy of tzdata for System.TimeZoneInfo, and Edge/IE also ship a copy of tzdata for the Intl implementation. But these might not be accessible from Go
• boost::time_zone handles this somehow in C++
• Windows 10 1709 added ucal_getDefaultTimeZone, ucal_openTimeZones etc in icuin.dll
• WinRT/UWP has Windows::Globalization::Calendar().GetTimeZone() available, that can be called by COM
• Use the classic Windows GetTimeZoneInformation C API and bundle in the (much smaller) mapping from IANA<-->Windows zone names

---

Actually while researching this, I discovered the %WINDIR%\Globalization\Time Zone\timezones.xml file seems to be a complete version of the tzdata database, using IANA names, in XML format :star:

From https://data.iana.org/time-zones/tz-link.html#CLDR i think this file was introduced in Windows 8.1. (As of January 2020 that will be the minimum Microsoft supported Windows version.)

I think adding support for parsing this file in time package would resolve this issue,

[leighmcculloch]

Changing the Windows logic to use %WINDIR%\Globalization\Time Zone\timezones.xml parallels what happens on Linux and Mac machines, by using the system values, so that sounds like it would make the most sense.

To preserve backwards compatibility would we need to have it still use Go's version if Go is installed, or would that be dangerous since developers running the app on the same operating system as a user who doesn't have Go installed may see different behavior?

[bradfitz]

I think adding support for parsing this file in time package would resolve this issue,

encoding/xml already depends on time, so making time depend on encoding/xml wouldn't be allowed. We'd need to do something tricky. Not impossible, but it'd be a little ugly.

[glycerine]

I favor using the identical zonefile on all builds. I don't want to deal with Heizenbugs that only affect windows. The windows .xml files don't have the Daylight saving time (DST) rules, for example.

[mappu]

I favor using the identical zonefile on all builds. I don't want to deal with Heizenbugs that only affect windows.

Mismatched zone files happen on Linux too, since the system version is used, and distros may be using a different or outdated system version of the tzdata database.

The windows .xml files don't have the Daylight saving time (DST) rules, for example.

That's not correct, the DST rules are present from line 2872 in my installation.

[ianlancetaylor]

Can someone show a sample of one of the files found on Windows? Thanks.

[mappu]

Can someone show a sample of one of the files found on Windows? Thanks.

%WINDIR%\Globalization\Time Zone\ directory from Windows 1903: Time Zone.zip

[ianlancetaylor]

Thanks. It does look to me as though the Windows data has all the information that the time package cares about. If that data is available on all or most Windows systems, it seems worthwhile to add support to the time package for reading it. As @bradfitz says above, the time package cannot use the encoding/xml package, but the data seems easy enough to parse.

Unfortunately, the fact that it is all in one big file means that fetching the information for a specific timezone will be slower than when reading the Unix-style tzdata files. But that is probably still a good tradeoff for Go programs running on Windows.

[thebitguru]

Would it be a short-term option to at least show a warning when compiling for Windows? That could save a lot of people a lot of time while the team decides on a proper long-term solution...

[ianlancetaylor]

I don't understand how that would help anybody. The great majority of Go programs use the time package, but very few need timezone information for any timezone other than the one they are in.

[elagergren-spideroak]

Since this hasn't seen love since in a few months (and has been kicking around for three years): I also have an application I need to distribute to Windows machines that very likely do not have the Go toolchain installed. My application uses LoadLocation.

[glycerine]

@elagergren-spideroak I have to hack the Go runtime with every new release because of this issue. https://github.com/glycerine/go/tree/timezone_windows_fix has the template for for the fix. It uses the approach from https://4d63.com/tz of embedding the needed data in the time package.

[ololosha228]

zoneinfo.zip is a mere 366776 bytes

@glycerine Wait a minute, is this? Okay, if so: why not just use windows api? may be it is, idn, cause i working on unixes, or may be not, but i pretty sure that windows os has any timezone api and timezone data.

As i can see, problem is not solved on two platforms: windows and linux (that's why i am here🙃🙃🙃🙃). Perhaps, it necessary to reopen this issue, cause it can be fatal problem on delivery any go apps on both platforms. @ianlancetaylor

[ianlancetaylor]

This issue is still open. There is a comment above suggesting that we can read the data from Windows. What needs to happen now is that somebody needs to write that code.

[glycerine]

@ololosha228 Please open a separate github issue if linux is having issues too. This one is focused on Windows. You can link to this if you think it is a similar issue, but I would guess it is different.

somebody needs to write that code.

We should just ship the "fat fix" of including the timezone file in the Go binary. Two+ years of this issue being open is depressing.

Later, as an optimization, windows binaries can be slimmed down if the 800KB becomes an issue. 100% prediction: it will never come up.

[ianlancetaylor]

Including zoneinfo.zip in every Go binary can be done but it's not a five minute job. The time package is at the bottom of the import hierarchy, and there are very few packages that it can import. In particular, it can't import archive/zip. So that code will have to be repurposed and slimmed down for use in the time package.

(And people do regularly complain about binary sizes, though perhaps not on Windows.)

[glycerine]

No need to unzip it (that would be complex and slow), just include it directly. Here's the diff of my patch. It is only 25 lines. And it adds one additional file with the data. Yes it is against an older version of Go, but the changes should still be trivial against head.

https://github.com/golang/go/compare/master...glycerine:timezone_windows_fix

[ololosha228]

@glycerine sure, i'll open new issue and try find way to fix it.

Btw, i have two questions:
1) why timezone data sized as chubby cat jpeg? I mean, 300+kb, for 24+ time zones... It's too large. But maybe i cam find explaination on stackoverlow, okay.
2) windows has timezone api, as i wrote above. May be for windows (not unix) it can be really good fix? Good reason is: you don't need delivery go dependency as timezones.zip (or how is it name).

I'll open issue for linux, and try to find same api on these weekends.

[ianlancetaylor]

@glycerine What is the effect on binary size of your patch?

[glycerine]

@ianlancetaylor I'll look when I'm back at the desk.

[glycerine]

@ianlancetaylor One of my DLLs goes from 68,247,552 bytes -> 67,714,048 bytes when the patch is removed. This is 0.7% change in size, a 533,504 byte difference.

Even if it was double that, since you mentioned earlier that might have happened in a subsequent addition, the fact that the binary can now RUN instead of inexplicably not run, it is so worth it.

Using _windows.go file extension, this could also be made windows-only.

[gopherbot]

Change https://golang.org/cl/224588 mentions this issue: time/tzdata: new package

[ianlancetaylor]

What do people think of the approach in https://golang.org/cl/224588? With that CL, any program that imports the new time/tzdata package (as import _ "time/tzdata") will get an embedded copy of the tzdata database. This will let the program work if the system does not have any timezone information. The program will first check the timezone information on the system, as they may be more up to date. If it doesn't find anything, it will use the embedded copy. This increases the size of the program by about 800K.

[elagergren-spideroak]

That sounds like a very pragmatic solution, thanks Ian.

[zikaeroh]

I think it should be up to the individual who deploys the binary to ensure tzdata is present.

With the _ import, if some library you depend on imports the tzdata package, then you're forced to take the 800K binary growth even if you plan to deploy with tzdata and would never use it. I can imagine some obscure library importing this ("we need timezone info, make sure we have it") and then importers who are unaware of this start growing as a result.

I'd personally rather see a flag to tell the compiler to include it than a package I can't opt-out of.

[glycerine]

What do people think of the approach in https://golang.org/cl/224588?

@ianlancetaylor I think that sounds great. Thank you for putting that together.

[glycerine]

I think it should be up to the individual who deploys the binary to ensure tzdata is present.

This is not realistic/too technical for a customer who installs your software. Welcome to the retail Windows software world.

[zikaeroh]

This is not realistic/too technical for a customer who installs your software. Welcome to the retail Windows software world.

This is not what I meant. I meant that at build time if you know you need it, you can include it in the binary. I am not saying that it's something extra that needs to be installed; that is the current solution.

Presumably, if you have the issue where you don't have a Go installation on the customer's machine, then you are also not asking them to compile it themselves (if you did, then tzdata would have been found). You're building for that customer and can enable some flag to ensure they have tzdata.

[alexbrainman]

With that CL, any program that imports the new time/tzdata package (as import _ "time/tzdata") will get an embedded copy of the tzdata database. ... This increases the size of the program by about 800K.

This will effectively means that all my programs will become 800K larger. Some developers of some package I use in my program will decide to include "time/tzdata", and I have to carry this useless extra disk space. This does not scale.

Alex

[leighmcculloch]

I think the CL (https://github.com/golang/go/issues/21881#issuecomment-602133486) is an elegant way to let a developer choose to bundle backup tzdata, but I don't think it brings timezone support on Windows to feature parity with other operating systems that Go supports.

For example, if a developer working on Linux or Mac builds and ships binaries on all operating systems, as is common, they won't find out there's a problem until they get their first Windows user who reports an issue with the application having no timezones. So there's still the element of surprise and the code we non-Windows users write is still not Windows friendly by default.

Because of this I think parsing in that XML file idea (https://github.com/golang/go/issues/21881#issuecomment-552239335) is still the best solution that's been suggested as it brings the time package for Windows into parity with other operating systems. It also keeps tzdata where we expect it, in the operating system.

[glycerine]

Some developers of some package I use in my program will decide to include "time/tzdata", and I have to carry this useless extra disk space.

No, you are not impotent in this scenario. Simply delete the import from the library.

I doubt this will pacify these voices, but let's add to the time/tzdata package documentation the convention that only the main package should be importing time/tzdata.

That way anyone can feel free to delete such an import if they find it in a library. No guilt.

[ianlancetaylor]

@leighmcculloch I think the time/tzdata suggestion is independent of adding support for parsing the Windows XML file. I think we should do that too. But even then there would be a use for some people for embedding the tzdata information in the binary itself.

[ololosha228]

What do people think of the approach in https://golang.org/cl/224588?

Yes, this is awesome platform-independent solution, i like it. Technically, it can be more graceful, but pragmatically, looks pretty.

But. I wonder that if this solution will release, some guys will describe it as _default_ solution, _not problem specific_. May be it is good to also edit docs? I can open pull request maybe.

---

Some developers of some package I use in my program will decide to include "time/tzdata", and I have to carry this useless extra disk space.

@alexbrainman is it? I agree, that extar 0.8MB is much space, but techically, you can cut imported package using cflags. but yes, every time use cflags is not a good idea. May be we can restrict use this package in others, except main?

[ianlancetaylor]

I've filed #38017 as a proposal for a time/tzdata package. Please comment or emoji vote there about this idea. Thanks.

[leighmcculloch]

I think the time/tzdata suggestion is independent of adding support for parsing the Windows XML file. I think we should do that too.

@ianlancetaylor I agree, sounds good! 👍

But even then there would be a use for some people for embedding the tzdata information in the binary itself.

For the use case of someone choosing to embed tzdata into their app, and not for solving this specific problem with Windows users, does it still make sense for the embedded tzdata to be only a backup? If I am choosing to embed data into an app I expect that data to be authoritative and be the only data used. I can see both cases (backup vs authoritative) being appropriate to different situations. Could we make the time/tzdata work like certificate pools and root CAs work? Any app can embed its own set of root CAs by creating a cert pool and adding that cert pool to the http package's default transport. The Go stdlib could still include the tzdata package, but I think the way it gets hooked up to the time package should allow:

1. Making it a backup source of tzdata
2. Making it an authoritative source
3. Specifying a custom source (unlikely to be used?)

[ianlancetaylor]

@leighmcculloch Can you discuss that over on the issue I just opened, #38017? Thanks.

[leighmcculloch]

Sorry @ianlancetaylor I think we posted about the same time. I'll post over on #38017.

[dmitshur]

Re-opening as CL 224588 was rolled back in CL 228200.

[gopherbot]

Change https://golang.org/cl/228101 mentions this issue: time/tzdata: new package

[mappu]

As per https://github.com/golang/go/issues/21881#issuecomment-602269937 , the new package in CL 228101 is not a fix for this issue - just a workaround that is not suitable in all cases.

Please re-open this ticket,

[ianlancetaylor]

I believe that the new time/tzdata package is a fix for this issue. And this issue has gotten lengthy and complex with 65 comments (now 66). Rather than reopening this issue, I encourage you to open a new issue for reading the XML file on Windows. Thanks.

[AndrewSav]

@mappu if you do, please link it here, so that those who are interested could follow

[ianlancetaylor]

@mappu opened the new issue at #38453. Thanks.