Openjdk-infrastructure: Windows machines need more than 100GB of disk space

Created on 8 Oct 2020  路  15Comments  路  Source: AdoptOpenJDK/openjdk-infrastructure

After playbook install, the amount of space used by various directories on the machines are as follows (Checked on one of the new test-ibmcloud-win2012r2 machines. This means there is barely 10GB of disk space for our processes to use and with the new 10GB minimum free disk space check we are hitting this limit. We should ensure that all Windows machines have more than this available - absolute minimum of 150Gb and ideally 250GB+. Build machines will likely require more.

  • cygwin_packages 2Gb
  • cygwin64 11.7Gb (11.2Gb in files)
  • Program Files 4.4GB
  • Program Files (x86) 36.5GB (36.2GB in files)
  • Windows 24.7Gb

Total of the above = 79.3Gb excluding other smaller directories on the machine

Breakdown of Program Files (x86) excluding a few directories which are under 1Gb:

  • LLVM 0.8Gb
  • Microsoft SDKs 1.3Gb
  • Microsoft Visual Studio 12.5Gb
  • Microsoft Visual Studio 9.0 0Gb
  • Microsoft Visual Studio 10.0 1.8Gb
  • Microsoft Visual Studio 11.0 0.9Gb
  • Microsoft Visual Studio 12.0 2.7Gb
  • Microsoft Visual Studio 14.0 2.2Gb
  • Windows Kits 11.5Gb
  • WiX Toolset v3.14 0.1Gb
enhancement
Source
picture sxa

Most helpful comment

@Willsparker @rajdeepsingh1 Good news - I've been able to run a sanity.openjdk run on a test machine and a JDK15/J9 build on the build box without issues. JDK8 failed due to an inconsistency in the JDK7 boot dir on the machine and how it was defined in jenkins which I've now resolved (permanant fix here) The build process does seem to use C:\workspace explicitly as part of the process but as long as that has enough space for one build (which it does) then that should be ok. We should definitely look at why the bulid fell back to trying to use --with-boot-jdk=C:openjdkjdk-8 as a default when JDK7 wasn't in the expected place and resolve that.

Copying @smlambert as there are new test-ibmcloud-win2012r2 machines in place now to replace test-softlayer-win2012r2 ones in case they show up anywhere any problems

picture sxa on 14 Oct 2020
🎉2 👍1

All 15 comments

Request to increase/reprovision new ibmcloud machines which have 100Gb has been sent to @rajdeepsingh1

sxa picture sxa on 8 Oct 2020

Sounds like adding a second disk to the system may be the easier option, so I'm happy with that - if we can get an extra disk to be used as the jenkins workspace drive that would meet the needs (subject to other stuff on the machines not expanding too much in the future!)

sxa picture sxa on 8 Oct 2020

With the new Cygwin install method, ( #1596 ), it should be substantially lower (Judging from a sub 2 minute install time). Those machines were setup with the cache install. I'm more than happy to remove that and do the new method?
Also, ref: #1594

Willsparker picture Willsparker on 8 Oct 2020

What is the size saving? But yes certainly happy for that to be done with the four new windows systems we're setting up

sxa picture sxa on 9 Oct 2020

After the new installation : cygwin64 is at 1.1G, cygwin_packages are at 167MB, so a good ~12GB saving

Willsparker picture Willsparker on 12 Oct 2020
👍1

Extra space will hopefully be available later today

sxa picture sxa on 13 Oct 2020

@sxa @Willsparker

I've added a 500GB disk to win2012r2-x86-1

This has been assigned as drive G: on the system. The Jenkins workspace is running on a 50G F: disk.

Please review and advise - I believe you would like the F: disk to be on the new 500G since it is the remoteFS: for Jenkins.

rajdeepsingh1 picture rajdeepsingh1 on 13 Oct 2020

Once this system is good to go I can add the additional 500G to the remaining 5 machines.

rajdeepsingh1 picture rajdeepsingh1 on 13 Oct 2020

Hi @rajdeepsingh1 - can you check if that's been applied to one of the AdoptOpenJDK systems as I can't seem to see the extra disk on them.

sxa picture sxa on 13 Oct 2020

@sxa @Willsparker

The following Windows machines should now have a 500G drive mounted and ready to go:

```
bld-ibm-w2012x1.adoptopenjdk.net
bld-ibm-w2012x2.adoptopenjdk.net
tst-ibm-w2012x1.adoptopenjdk.net
tst-ibm-w2012x2.adoptopenjdk.net

rajdeepsingh1 picture rajdeepsingh1 on 14 Oct 2020
👍2

~@Willsparker Can you verify that t Jenkins user can write to the new drives then we can adjust the Jenkins definitions to point the workspace onto it~
I've adjusted the definitions on the machines to point at the new drives and set permissions accordingly - testing now

sxa picture sxa on 14 Oct 2020

Thanks @sxa - can you link to the tests?

Willsparker picture Willsparker on 14 Oct 2020

All working now - I was just running an echo to show that it was running from the correct location so we can run some proper tests on them now :-D I'll close this issue since the space issue is now resolved

sxa picture sxa on 14 Oct 2020
👍1

@Willsparker @rajdeepsingh1 Good news - I've been able to run a sanity.openjdk run on a test machine and a JDK15/J9 build on the build box without issues. JDK8 failed due to an inconsistency in the JDK7 boot dir on the machine and how it was defined in jenkins which I've now resolved (permanant fix here) The build process does seem to use C:\workspace explicitly as part of the process but as long as that has enough space for one build (which it does) then that should be ok. We should definitely look at why the bulid fell back to trying to use --with-boot-jdk=C:openjdkjdk-8 as a default when JDK7 wasn't in the expected place and resolve that.

Copying @smlambert as there are new test-ibmcloud-win2012r2 machines in place now to replace test-softlayer-win2012r2 ones in case they show up anywhere any problems

sxa picture sxa on 14 Oct 2020
🎉2 👍1

@sxa Awesome, thanks :-)

Willsparker picture Willsparker on 15 Oct 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

lumpfish picture lumpfish  路  4Comments
M-Davies picture M-Davies  路  9Comments
sxa picture sxa  路  3Comments
M-Davies picture M-Davies  路  4Comments
smlambert picture smlambert  路  4Comments