Temurin-build: Add nightly build support for BishengJDK-11 and BishengJDK-8

[kuenking]

Hi, I'm a member of the BishengJDK development team from Huawei. We are very interested in building and distributing binaries of BishengJDK via AdoptOpenJDK.

BishengJDK-8 aarch64 port was based on aarch64-port/jdk8u-shenandoah/ and BishengJDK-11 aarch64 port was based on jdk-updates/jdk11u/. Here is our open-sourced repo on Gitee:
BishengJDK-8: https://gitee.com/openeuler/bishengjdk-8 and BishengJDK-11: https://gitee.com/openeuler/bishengjdk-11. For more information, please refer to the WIKI.
Mirror on Github : https://github.com/openeuler-mirror/bishengjdk-11 and https://github.com/openeuler-mirror/bishengjdk-8

How should we get started? Is there anything we can do?

Look forward to your reply. thank you .

[aahlenst]

@kuenking Great news. I listed the most important points already in https://github.com/AdoptOpenJDK/openjdk-build/issues/2420#issuecomment-769747684. What you can do immediately is make the build scripts here compatible with Bisheng and then connect with the rest of the team to enable the pipelines. Might make sense to sign-up on Slack.

Some of the work for Bisheng was already done as part of https://github.com/AdoptOpenJDK/openjdk-build/pull/2485/files.

[kuenking]

@aahlenst
Thank you for your acceptance and suggestions, we will launch a pr as soon as possible, thank you！

[sxa]

With https://github.com/AdoptOpenJDK/openjdk-build/pull/2523 having gone in to change the repo we pull the Bisheng code for RISC-V to the same one you mention above I'm reasonably sure this will be very easy to implement with minimal work :-)

[sxa]

@kuenking Should those platforms be build from the master branch of the repositories? We've currently fixed bisheng to be the risc-v branch and the JDK11 repo is hardcoded but those are set here for the branch and here for the repository so easy enough to fix :-)

Is this only Linux/aarch64 that is required?

[sxa]

I've had a quick test and the build scripts seem capable of building from those repositories on x64 and aarch64. It will also need an adjustment to getOpenJdkVersion in build.sh to handle your JDK8 repository, correctly but otherwise builds with no significant problems in my development environment (I tested without debug symbols):

=JAVA VERSION OUTPUT=
openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment Bisheng (build 11.0.10+9-202103181725)
OpenJDK 64-Bit Server VM Bisheng (build 11.0.10+9-202103181725, mixed mode)
=/JAVA VERSION OUTPUT=

=JAVA VERSION OUTPUT=
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment Huawei (build 1.8.0-internal-202103181751-b00)
OpenJDK 64-Bit Server VM Huawei (build 25.71-b00, mixed mode)
=/JAVA VERSION OUTPUT=

[andrew-m-leonard]

If consistent java version output is required, there needs to be a defined set of git "tags" which the build scripts determine the jdk build level from, or a version.txt file, like: https://github.com/feilongjiang/bishengjdk-11-mirror/blob/risc-v/version.txt
What is the theme for tags in : https://github.com/openeuler-mirror/bishengjdk-11/tags ? not sure I can see a consistent tagging theme?

[sxa]

FYI @kuenking I think one of the kae commits from yesterday to the JDK8 repository has resulted in a compilation error. Thought I'd done something wrong but it seems to build ok again if I go back to the last ga tag :-)

[feilongjiang]

What is the theme for tags in : https://github.com/openeuler-mirror/bishengjdk-11/tags ? not sure I can see a consistent tagging theme?

@andrew-m-leonard @sxa We have two branches in bishengjdk-11: master (for aarch64) and risc-v (for riscv64). The tag named as bisheng_riscv is the release tag for risc-v, and others for master. We have discussed that maybe use version.txt file is a better way to set the java version. We will add a version.txt file for each branch.

[kuenking]

@sxa @andrew-m-leonard
1、I'm sorry I didn't respond to the message in time for anything else.
2、For AArch64, add --disable-warnings-as-errors to JDK11 and --with-extra-cxxflags="-Wno-error" --with-extra-cflags="-Wno-error" to JDK8.
3、We have discussed that maybe use version.txt file is a better way to set the java version. PR has been integrated into the Gitee for Bisheng JDK. One day later, Github for Bisheng will complete code synchronization.
4、for bisheng jdk11 , version.txt 11.0.10.0.12 -----> build 11.0.10+12
for bisheng jdk8, version.txt 8.282.8.0.12 -----> build 1.8.0_282-b12

Thanks all

[sxa]

Thanks. If you are able to add in a version.txt into each branch that would make things a lot less complex in the build scripts so that would definitely be my preferred option. I'll hold of on completing this pull request until we have those files in place

[eapenzhan]

@sxa the version.txt has been merged into the github mirror of bisheng jdk.
https://github.com/openeuler-mirror/bishengjdk-11
https://github.com/openeuler-mirror/bishengjdk-8
You may now change the build scripts as you planned.
Great thx!

BTW, I'm a member of bisheng team.

[sxa]

@kuenking @feilongjiang I'm getting the following on JDK8/aarch64 but not on JDK8/x64 - is there a known build break in the code base just now? This is with GCC 7.5

In file included from /home/jenkins/openjdk-build/build-farm/workspace/build/src/jdk/src/solaris/native/org/openeuler/security/openssl/kae_cipher_aes.c:24:0:
/usr/include/openssl/evp.h:652:5: note: expected 'int *' but argument is of type 'unsigned int *'
 int EVP_CipherFinal_ex(EVP_CIPHER_CTX *ctx, unsigned char *outm, int *outl);
     ^~~~~~~~~~~~~~~~~~
/home/jenkins/openjdk-build/build-farm/workspace/build/src/jdk/src/solaris/native/org/openeuler/security/openssl/kae_cipher_rsa.c: In function 'Java_org_openeuler_security_openssl_KAERSACipher_nativeCreateRSAPrivateCrtKey':
/home/jenkins/openjdk-build/build-farm/workspace/build/src/jdk/src/solaris/native/org/openeuler/security/openssl/kae_cipher_rsa.c:301:9: warning: implicit declaration of function 'RSA_set0_key'; did you mean 'RSA_check_key'? [-Wimplicit-function-declaration]
     if (RSA_set0_key(rsa, bnN, bnE, bnD) <= 0 ||
         ^~~~~~~~~~~~
         RSA_check_key
/home/jenkins/openjdk-build/build-farm/workspace/build/src/jdk/src/solaris/native/org/openeuler/security/openssl/kae_keypairgenerator_rsa.c:49:5: error: 'RSA_get0_n' undeclared here (not in a function); did you mean 'RSA_verify'?
     RSA_get0_n,
     ^~~~~~~~~~
     RSA_verify

[eapenzhan]

@sxa, could you tell me the version of openssl in your aarch64 environment. It requires at least 1.1.1a to build, which might be the problem, as it shows that it didn't find the function RSA_set0_key and RSA_get0_n belonging to openssl lib.

We use openssl lib in aarch64 version, but not in x64. That's the reason the x64 build is ok.

[sxa]

@sxa, could you tell me the version of openssl in your aarch64 environment. It requires at least 1.1.1a to build, which might be

That would likely explain it - none of our other builds require that so far. We build on CentOS7 which has 1.0.2k.

[sxa]

For the vesion.txt parsing I'm currently using:

      if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then
        local updateNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk8u${updateNum}-b${buildNum}"
      else
        local minorNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local updateNum="$(cut -d'.' -f 3 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk-11.${minorNum}.${updateNum}+${buildNum}"
      fi

Not sure if those are the values you want - in particular I'm not sure on the value you'd like at the end of jdk8u282- or if you want a different format for your other segments of version.txt

[eapenzhan]

For the vesion.txt parsing I'm currently using:

      if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then
        local updateNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk8u${updateNum}-b${buildNum}"
      else
        local minorNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local updateNum="$(cut -d'.' -f 3 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk-11.${minorNum}.${updateNum}+${buildNum}"
      fi

Not sure if those are the values you want - in particular I'm not sure on the value you'd like at the end of jdk8u282- or if you want a different format for your other segments of version.txt

You're so cool. That's exactly what we want!

[eapenzhan]

@sxa, could you tell me the version of openssl in your aarch64 environment. It requires at least 1.1.1a to build, which might be

That would likely explain it - none of our other builds require that so far. We build on CentOS7 which has 1.0.2k.

Is it posssible that you could update the openssl version of the image? Otherwise, we have to disable the building of the feature as x64.

[feilongjiang]

For the vesion.txt parsing I'm currently using:

      if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then
        local updateNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk8u${updateNum}-b${buildNum}"
      else
        local minorNum="$(cut -d'.' -f 2 <${bishengVerFile})"
        local updateNum="$(cut -d'.' -f 3 <${bishengVerFile})"
        local buildNum="$(cut -d'.' -f 5 <${bishengVerFile})"
        version="jdk-11.${minorNum}.${updateNum}+${buildNum}"
      fi

Looks ok to BishengJDK-11 risc-v. ;-)

[sxa]

Is it posssible that you could update the openssl version of the image? Otherwise, we have to disable the building of the feature as x64.

I tried some options yesterday uit couldn't make it work. I put a copy of 1.1.1j in another directory but I could not make the bisheng build process pick it up. Of the options below I world prefer the first one on this list of possible:

• Have a configure option on your build to say where to pick up the openssl header files from (NOTE: This is what OpenJ9 uses - they have a configure parameter --with-openssl=/usr/local/openssl-x.y.z which we use
• Use a general option such as CLAGS and LDFLAGS to point the build at the location of 1.1.1 (I tried a few options but couldn't make that work yesterday)
• Build on a different distribution which has OpenSSL 1.1.1 by default, but It's rather not have a different one just for Bisheng on aarch64
• If there is an option for it in the build, disable OpenSSL for now until we figure out the best solution :-)
• Trying to replace the default OpenSSL would likely have other problems as the automatic patching and dependency management on the system would not be able to occur and on a quick test yesterday type such as curl on the machine did not work with the system OpenSSL being upgraded outside the package manager

It may still be possible to find a solution to the last one but if there was a way to direct it to another location for the openssl give that world be my preferred option.

[eapenzhan]

@sxa, thank you for your effort. We use LD_LIBRARY_PATH to specify the openssl lib dir. You could have a try.
If it still doesn't work, add --disable-kae option when configuring to disable building the feature.

Some details:

• LD_LIBRARY_PATH is a environment variable
• The option is applied in this way, bash configure --disable-kae

[sxa]

With --disable-kae it suceeds:

=JAVA VERSION OUTPUT=
openjdk version "1.8.0_282-internal"
OpenJDK Runtime Environment Huawei (build 1.8.0_282-internal-202103251619-b12)
OpenJDK 64-Bit Server VM Huawei (build 25.282-b12, mixed mode)
=/JAVA VERSION OUTPUT=

Just setting LD_LIBRARY_PATH didn't work (as that only affects runtime, not the linker process during the build) however if I use the full set of --with-extra-cflags=-I/usr/local/openssl-1.1.1/include --with-extra-cxxflags=-I/usr/local/openssl-1.1.1/include --with-extra-ldflags=-L/usr/local/openssl-1.1.1/lib to point to the 1.1.1 version then the build completes without problems. I've added that into my PR at https://github.com/AdoptOpenJDK/openjdk-build/pull/2537/files. With all these tests I'm glad I'm using an aarch64 system that can build in 5 minutes ;-)

Thank you for the advice :-)