Temurin-build: How to update timezone data with AdoptOpenJDK

[keirlawson]

I have been trying to update timezone data within the latest AdoptOpenJDK 11 image, however Oracle's tzupdater does not appear to work, crashing with a null pointer exception. Is there a recommended way to update timezones with AdoptOpenJDK?

[karianna]

Does this happen with an Adopt binary outside of Docker? What error message do you see?

[keirlawson]

Yes I am able to recreate this issue on my machine outside of docker, the command output (with the verbosity flag on) is as follows:

$ java -jar resources/tzupdater.jar --location --force -v
Using https://www.iana.org/time-zones/repository/tzdata-latest.tar.gz as source for tzdata bundle.
java.home: /Library/Java/JavaVirtualMachines/adoptopenjdk-12.jdk/Contents/Home
java.vendor: AdoptOpenJDK
java.version: 12
tzupdater version 2.2.0-b01
JRE tzdata version: tzdata2018g
Downloaded file to /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tz.tmp/tzdata.tar.gz
java.lang.NullPointerException
Exception in thread "main" com.sun.tools.tzupdater.TzRuntimeException: java.lang.NullPointerException
    at com.sun.tools.tzupdater.TimezoneUpdater.main(TimezoneUpdater.java:653)
Caused by: java.lang.NullPointerException
    at com.sun.tools.tzupdater.TimezoneUpdater.run(TimezoneUpdater.java:215)
    at com.sun.tools.tzupdater.TimezoneUpdater.main(TimezoneUpdater.java:634)

Not sure why its picking up my Java version as being 12 as:

$ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.2+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.2+9, mixed mode)
$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home

[karianna]

@keirlawson Ah looks like you're running on a Mac. Apple has a special internal tool which sets the default JAVA_HOME etc.

I use a shell script to swap between them:

# List all of the Java's that are available
alias java_ls='/usr/libexec/java_home -V 2>&1 | grep -E "\d.\d.\d[,_]" | cut -d , -f 1 | colrm 1 4 | grep -v Home'

# Swap between Java's
function use_java() {
    export JAVA_HOME=$(/usr/libexec/java_home -v $1)
    export PATH=$JAVA_HOME/bin:$PATH
    java -version
}

[karianna]

But yeah the NPE is annoying - we'll have to dig into this - not a docker problem though so I'll shift this.

[ali-ince]

Digging into this a bit, I've found out couple of things to note;

Oracle TZ Updater has this following information on its download page;

System Requirements

The TZUpdater tool supports all currently supported versions of the Oracle JDK and JRE, on all supported platforms. The java.vendor property value must be Sun Microsystems Inc. or Oracle Corporation or BEA Systems,Inc.

Which makes me think that it won't work anyways.

There's also another tool from Azul Systems (available at https://www.azul.com/products/open-source-tools/ziupdater-time-zone-tool/), which is released under GPLv2 but fails to execute under jdk11 with a text of [ziupdater]unsupported Java version 11.0.3.

If I'm not missing any information, I think we need to discuss our options here.

[karianna]

Interesting - do we access to Azuls source? Hopefully we can fork/patch
that to work with Adopt

On Sat, 27 Apr 2019 at 12:13, Ali Ince [e-mail] wrote:

Digging into this a bit, I've found out couple of things to note;

Oracle TZ Updater has this following information on its download page;

System Requirements

The TZUpdater tool supports all currently supported versions of the Oracle
JDK and JRE, on all supported platforms. The java.vendor property value
must be Sun Microsystems Inc. or Oracle Corporation or BEA Systems,Inc.

Which makes me think that it won't work anyways.

There's also another tool from Azul Systems, which is released under GPLv2
but fails to execute under jdk11 with a text of unsupported version.

If I'm not missing any information, I think we need to discuss our options
here.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/AdoptOpenJDK/openjdk-build/issues/1057#issuecomment-487277125,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABME2D2GVKBSNWHNGDOXOLPSQYNFANCNFSM4HIV2ISA
.

>

Cheers, Martijn (Sent from Gmail Mobile)

[keirlawson]

Yea I found the Azul one, but couldn't find the source anywhere despite it supposedly being open source. Perhaps if someone has a contact at Azul they might be able to get it out of them?

[ali-ince]

I am still not sure whether Azul's ZIUpdater is released with source code or not, because it has this info at the bottom of the page I linked before

Please send us questions / problems / source snapshot requests at [email protected]

But theoretically we can ask for the source codes. Do we have contacts at Azul @karianna?

[karianna]

Yes I'll ping them now.

[keirlawson]

It looks like OpenJDK may already have a utility to solve part of the problem, though not widely publicised: https://github.com/akashche/tzdbgen

[SueChaplain]

IBM has a TZUpdater. Spoke briefly to Tim about options to open source it. We are investigating.

[karianna]

Azul is also discussing this

[casper5822]

Hello, is there any new about this problem?

Thank you

[karianna]

@SueChaplain Any news from your side? I'll chase Azul formally.

[karianna]

Have sent formal request into Azul

[casper5822]

I tried also the other tool suggested by @keirlawso. I use adoptopenjdk 12 and it seems working but is not clear how to pass the extra tz files related to openjdk version. So the tool uses only the standard files so it has an incomplete documentation or functionalities. I opened also a github issue for this tool.

[SueChaplain]

@karianna - discussions still ongoing here to confirm originality of code. Will prod.

[casper5822]

Hope a tzupdater will be soon available, we have a critical timezone cloud platform, with payments and invoices, timezone is a vital part of our microservices.

[casper5822]

Just to inform the community:
The Azul tzupdater works only with adoptopenjdk 8.
I tried it with our docker containers with the last iana db: 2019a with this version :

https://web.cs.ucla.edu/~eggert/tz/release

The offical IANA tar.gz dbs contain bugs, so we use this one.
So for now, to fix this problem, we have to downgrade to java 8. (Timezone are more important than java version itself in our case).

Maybe Azul can check the support for java 11 and 12, so that the tool can work correctly with adoptopenjdk, without make big changes or overwrite lot of code.

[muyinliu]

It looks like OpenJDK may already have a utility to solve part of the problem, though not widely publicised: https://github.com/akashche/tzdbgen

see also: ZoneRulesBuilder.java in path /make/src/classes/build/tools/tzdb/ of jdk8

[keirlawson]

Is there any update on either the Azul or IBM solutions?

[karianna]

@karianna - discussions still ongoing here to confirm originality of code. Will prod.

Hi Sue,

Did you have any further luck?

[paulocesarcuneo]

This seems to work on zulu, openjdk & corretto (or at least it doesn't throw, didn't test dates, nor timezones).

https://bell-sw.com/pages/iana-updater/

It uses the "rearguard" format, though; so you must build the tzdata.tar.gz yourself.

git clone https://github.com/eggert/tz
git checkout 2019b
make tarballs

java -jar IANAUpdater.jar -t ${JAVA_HOME} -z  tzdata2019b-rearguard.tar.tz

Last version of oracle tzupdater work with openjdk, but not with zulu nor corretto.
https://github.com/akashche/tzdbgen seems abandoned, and it throws.
ziupdater from azul also throws.
But copy/pasting build.tools.tzdb from java.net/openjdk into a proyect seems to work.

All java "tz-parser" use the "rearguard" format. But IANA only publishes "vanguard" format.

[muyinliu]

@paulocesarcuneo

Please execute make rearguard_tarballs instead of make in folder tzdb-xxx(IANA Time Zone Database Complete Distribution) to generate valid input for tzdbgen, TZUpdater or IANAUpdater

[SueChaplain]

@karianna - @andrew-m-leonard and team have been taking a look and can let you know how that's going.

Hi Sue,

Did you have any further luck?

[groeges]

The IBM Timezone Updater currently only supports updates being made to IBM Java 1.4.2, 5.0, 6, 7 and 8. It does not support updates being made to OpenJDK8 or OpenJDK9+ (for both Hotspot or OpenJ9 variants). We are currently investigating what needs to be done in order to make the Timezone Updater work with these versions. Will keep you posted on progress of this work.

[djphillyp]

Hi it seems that the Azul ziupdater jar version 1.0.2.2 now seems to correctly update the timezone data, when using a rearguard formatted 2019b timezone file, with Java 11

[andrew-m-leonard]

@djphillyp Does the Azul updater work for jdk8 as well?

[andrew-m-leonard]

Just read the doc! it does support jdk8 :-)

[davehobbs]

@karianna As Azul already have a publicly available timezone updater which works with Java 11 is there any reason to use the IBM Timezone Updater? The only function which the IBM tool has which is not in the Azul package is the ability to update a set of Java installs on a hard drive from a settings file and optionally generate the settings file using a set of include/exclude criteria.

[davehobbs]

For anyone wanting a later version of the 'rearguard' format, you can build your own. See https://stackoverflow.com/questions/56908541/update-to-tzdata2019b-is-failing-tzupdater-version-2-2-0-b01

[karianna]

@karianna As Azul already have a publicly available timezone updater which works with Java 11 is there any reason to use the IBM Timezone Updater? The only function which the IBM tool has which is not in the Azul package is the ability to update a set of Java installs on a hard drive from a settings file and optionally generate the settings file using a set of include/exclude criteria.

I wasn't aware this was publicly available? I had to manually request the source code from them...

[davehobbs]

Publicly available binary. Do we need the source?

[karianna]

Publicly available binary. Do we need the source?

Yes, we need the source :-) - Adopt policy not to have mystery meat builds.

[djphillyp]

I might be wrong but could this code not be wrapped around to provide the functionality?

https://github.com/openjdk/jdk/tree/0c3f9c6012a5878bc9275a5abad6b3d044a30ba7/make/jdk/src/classes/build/tools/tzdb

I also notice that there was a recent commit to that to support the "vanguard" format, which would simplify things too as the files published by IANA could be used directly again!

[webjoel]

@paulocesarcuneo

Please execute make rearguard_tarballs instead of make in folder tzdb-xxx(IANA Time Zone Database Complete Distribution) to generate valid input for tzdbgen, TZUpdater or IANAUpdater

Result:

make: * No rule to make target 'rearguard_tarballs'. Stop.

[sgrinev]

@paulocesarcuneo
Result:
make: * No rule to make target 'rearguard_tarballs'. Stop.

make sure you've downloaded the full bundle with sources. Next has just worked for me:

wget https://data.iana.org/time-zones/releases/tzdb-2019b.tar.lz
lzip -d tzdb-2019b.tar.lz
tar xf tzdb-2019b.tar
cd tzdb-2019b/
make rearguard_tarballs

[webjoel]

@paulocesarcuneo
Result:
make: * No rule to make target 'rearguard_tarballs'. Stop.

make sure you've downloaded the full bundle with sources. Next has just worked for me:

wget https://data.iana.org/time-zones/releases/tzdb-2019b.tar.lz
lzip -d tzdb-2019b.tar.lz
tar xf tzdb-2019b.tar
cd tzdb-2019b/
make rearguard_tarballs

Same error. :(

[sgrinev]

@paulocesarcuneo then something is different with your environment. Try docker run ubuntu

[webjoel]

@paulocesarcuneo @sgrinev My problem is Ubuntu 16.04, solution:

make AWK=gawk rearguard_tarballs

[jmullee]

java -Djava.vendor="Oracle Corporation" -verbose:class -jar "${PWD}/tzupdater.jar" -v -l "file://${PWD}/${RGTZ}"

[andrew-m-leonard]

fyi, https://github.com/AdoptOpenJDK/TSC/issues/123

[CaeirOps]

This way have solved my problem:

sudo java -Djava.vendor="Oracle Corporation" -jar "${PWD}/tzupdater.jar" -l

[JigarJoshi]

any update here?

[JigarJoshi]

Today it broke the systems again Oracle's tzupdater at version 2.3.1 doesn't like this new release of 2020b and fails

https://stackoverflow.com/questions/64254417/tzupdate-failures-with-2020b

May be this is the right time to pull it off

[andrew-m-leonard]

@JigarJoshi It looks like Oracle have published version 2.3.2, which given the bug has now been marked as resolved, I would hope fixes this issue.

[karianna]

@keirlawson can you give it a try as well?

[JigarJoshi]

@andrew-m-leonard yes the issue is fixed with 2.3.2.

[JigarJoshi]

Do you think it is still good idea to add OpenJDK version of timezone updater tool?

[karianna]

Do you think it is still good idea to add OpenJDK version of timezone updater tool?

It would be great - I think the best thing would be to ask Oracle very politely if they'd consider it.