Temurin-build: docker builds failing

[gdams]

example failure: https://ci.adoptopenjdk.net/job/openjdk8_build_docker/190/console

ls: cannot access 'openjdk-copy-src:/openjdk/build/': No such file or directory
Updating source
Error response from daemon: Container 00cfe99f669cb019e18786c09391152ec17c7d132abd38358011ce9c498cff06 is not running
Shutting down
openjdk-copy-src
rm: cannot remove 'docker/jdk8u/x86_64/ubuntu/sbin': No such file or directory
Building as you've not specified -k or --keep

Building docker container
unable to prepare context: path "docker/jdk8u/x86_64/ubuntu" not found

Unable to find image 'openjdk_container:latest' locally
docker: Error response from daemon: repository openjdk_container not found: does not exist or no pull access.
See 'docker run --help'.

[gdams]

@adamfarley this looks like it was broken by the renaming of the docker files? Is this something that can be fixed?

[gdams]

fixed by #197

[stevewallin]

@gdams I still have jdk8u and jdk 9 docker builds failing with:

bash: ./configure: No such file or directory

Failed to configure the JDK, exiting
Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR
For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64
Current JDK_BOOT_DIR value: /usr/lib/jvm/java-1.8.0-openjdk-amd64

Looks like the source tree is not available inside the container

[adamfarley]

I'm seeing docker failures on the adoptopenjdk jenkins page.

e.g. https://ci.adoptopenjdk.net/job/openjdk9_build_docker/118/console

The failures all seem to start with this:

Configuring command and using the pre-built config params...
Currently at '/openjdk/build/openjdk'
Not reconfiguring due to the presence of config.status in /openjdk/build

[gdams]

@stevewallin what command are running? Perhaps if I can recreate your issue I can work out what is going wrong

[stevewallin]

Working now... I was having strange issues with the container storage throwing errors so I have cleaned everything out and now have a complete run with bash makejdk-any-platform.sh -v jdk8u
It now completes ok, but there are still some additional "/" in some config paths that might need fixing up.
Completed configuring the version string parameter, config args are now: --with-boot-jdk=/usr/lib/jvm/java-1.7.0-openjdk-amd64 --enable-ccache --with-jvm-variants=server --with-cacerts-file=/openjdk/build/cacerts_area/security/cacerts --with-alsa=/openjdk/build/alsa-lib-1.0.27.2 --with-freetype=/openjdk/build//openjdk/installedfreetype --with-x=/usr/include/X11 --with-debug-level=release

Built the JDK!

/openjdk/build//openjdk/build/linux-x86_64-normal-server-release/images/j2sdk-image' found

[karianna]

@adamfarley You still seeing failures?

[adamfarley]

Yep, still seeing failures.

The runs can be found here:

https://ci.adoptopenjdk.net/job/openjdk9_build_docker/
https://ci.adoptopenjdk.net/job/openjdk9_openj9_build_docker/

It fails fairly quickly, runs regularly, and can be kicked off manually by anyone with access.

[gdams]

The issue is due to the docker containers not getting properly cleaned up:
Not reconfiguring due to the presence of config.status in /openjdk/build

[neomatrix369]

I will say I have also seen something similar happen https://github.com/AdoptOpenJDK/openjdk-build/issues/194#issuecomment-350673135 or even https://github.com/AdoptOpenJDK/openjdk-build/issues/194#issuecomment-350692988 when running in a native environment, and now it looks like it's happening on more environments/machines.

Someone also mentioned about this on slack, might need a good sanity check to see our scripts have _referential transparency_ (always gives the same results when run each time, in short).

[neomatrix369]

Here's the usage of configure for OpenJDK9 (might not be the latest, so please check on your end as well) - https://gist.github.com/neomatrix369/2aa4ee11e2a459f18ac271868b2078bc - I see there's an option to create the config in a different directory, might be worth doing that and flushing that folder each time we do the build in docker - just an idea, to avoid one of these errors.

when running inside docker better to run with --debug-configure to see if we get some clues to why the configure part fails intermittently.

[karianna]

I have the docker build complaining about:

No parameters provided, using Docker.

We'll remove the built Docker container if you're using Docker.

BRANCH is undefined so checking out dev
WORKING_DIR is undefined so setting to /Users/karianna/Documents/workspace/AdoptOpenJDK_Projects/openjdk-build.

TARGET_DIR is undefined so setting to /Users/karianna/Documents/workspace/AdoptOpenJDK_Projects/openjdk-build
If you're using Docker the build artifact will not be copied to the host.
Cloning OpenJDK git repo

origin  https://github.com/breadbin/openjdk-jdk8u.git (fetch)
origin  https://github.com/breadbin/openjdk-jdk8u.git (push)
Will reset the repository at /Users/karianna/Documents/workspace/AdoptOpenJDK_Projects/openjdk-build/openjdk in 10 seconds...
Pulling latest changes from git repo
Git repo fetch mode: shallow (DOES NOT contain commit history)
Fetching origin
Username for 'https://github.com': karianna
Password for 'https://[email protected]':
remote: Repository not found.
fatal: repository 'https://github.com/breadbin/openjdk-jdk8u.git/' not found
error: Could not fetch origin
HEAD is now at bfeacb81 Attribute usage of FreeType

I have no idea where breadbin is coming from, can't see it in any scripts.

[karianna]

OK, it got past that warning and eventually died on:

/openjdk/build/cacerts_area/security/cacerts: Java KeyStore
Successfully retrieved the cacerts file!

real    0m2.852s
user    0m0.200s
sys 0m0.080s
fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
OpenJDK repo tag is
Completed configuring the version string parameter, config args are now:  --with-version-pre=adoptopenjdk --with-version-string=
Building up the configure command...
Overriding JDK_BOOT_DIR, set to /usr/lib/jvm/java-1.7.0-openjdk-amd64
Boot dir set to /usr/lib/jvm/java-1.7.0-openjdk-amd64
Completed configuring the version string parameter, config args are now:  --with-version-pre=adoptopenjdk --with-version-string= --with-boot-jdk=/usr/lib/jvm/java-1.7.0-openjdk-amd64 --enable-ccache --with-jvm-variants=server --with-cacerts-file=/openjdk/build/cacerts_area/security/cacerts --with-alsa=/openjdk/build/alsa-lib-1.0.27.2 --with-freetype=/openjdk/build//openjdk/installedfreetype --with-x=/usr/include/X11 --with-debug-level=release
Should have the source, I'm at /openjdk/build/openjdk
Configuring command and using the pre-built config params...
Currently at '/openjdk/build/openjdk'
Running ./configure with arguments ' --with-version-pre=adoptopenjdk --with-version-string= --with-boot-jdk=/usr/lib/jvm/java-1.7.0-openjdk-amd64 --enable-ccache --with-jvm-variants=server --with-cacerts-file=/openjdk/build/cacerts_area/security/cacerts --with-alsa=/openjdk/build/alsa-lib-1.0.27.2 --with-freetype=/openjdk/build//openjdk/installedfreetype --with-x=/usr/include/X11 --with-debug-level=release  '
Running generated-configure.sh
configure: error: unrecognized options: --with-version-pre, --with-version-string
configure exiting with result code 1

So it doesn't understand --with-version-pre=adoptopenjdk --with-version-string= that's being passed to it.

[karianna]

Note that this path of executing configure from /openjdk/build/openjdk clashes with the way it needs to be run for #274 (from /openjdk/build) - I suspect the two methods can be unified.

[JamesKingdon]

I tried ./makejdk-any-platform.sh --version jdk9 -bv openj9 which behaves very strangely. It seems to do a fair amount of building docker images and installing cacerts. Then there's a series of error messages:

fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: Not a git repository (or any parent up to mount point /openjdk/build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
OpenJDK repo tag is 
Completed configuring the version string parameter, config args are now:  --with-version-pre=adoptopenjdk --with-version-string=

I get the Not reconfiguring due to the presence of config.status in /openjdk/build message, then it does something which doesn't take long enough to be an actual build and claims to put the result in /openjdk/target, where I find a version of OpenJDK/Hotspot with a December 2017 date!

[karianna]

yep, existing scripts are pretty broken :-|. Working on the fix!

[JamesKingdon]

Thanks Martijn. Let me know if there's anything I can do to help.

[karianna]

We need to re-add these to the pipelines.

[M-Davies]

@karianna What's the status on this? Looking at our current docker setup, it looks very different than what was discussed 2017-2018?

[karianna]

So the docker builds are still failing but I think we should be raising a new specific issue(s).