Openj9: AIX build didn't relocate out of the way of compressed heap

[zl-wang]

By default, AIX 64bit program text and data are loaded/located in segments starting at 0x1,0000,0000 (i.e 4GB effective address). That interferes with compressed heap allocation of anything bigger than 3GB. It also limits the growth of malloc space (limited to grow to meet the compressed heap allocation). That literally defeats each other.

Since AIX6.1, -T or -bpT option of ld can be used to specify an alternative text/data location. We can get out of the way of compressed heap allocation. For example, -bpT:0x10000000000 option can be used to do that. That ld option is specifying text/data to be located at effective address starting at 0x100,0000,0000 (1TB).

[zl-wang]

This issue should be applicable to all builds on AIX. I found java8/11 LTS text/data located at 4GB.

[pshipton]

@zl-wang which shared libraries or executables need to use the option? This seems it could affect more than the JVM build, AIX should change their defaults.

Related to #7115 - Java Launcher is loaded at 0x80000000 on zLinux

[zl-wang]

@pshipton only the final executable needs to use that option, i.e. java. there is a similar option for Linux. we used them both (AIX and Linux) before. however, my test on Linux LE with OpenJ9 driver seemed showing text/data being out of the way already. so that, I didn't mention it in this issue.

[zl-wang]

on both AIX and Linux. all shared library are located by default far away from the top (low-end) address.

[pshipton]

@andrew-m-leonard @M-Davies fyi that AIX needs a similar fix as #7115

[pshipton]

What is the location used for IBM Java 8?

[zl-wang]

you meant the launcher text? it has been always 0x100,0000,0000 (i.e. !TB address).
plus, a reminder: the linker option can only accept values between 1TB and (1TB+256MB).

[M-Davies]

I found this

jdk/make/CompileLaunchers.gmk
 362   ifeq ($(OPENJDK_TARGET_OS), aix)
 363     ifeq ($(OPENJDK_TARGET_CPU_BITS), 32)
 364       JAVAW_LDFLAGS  += -bmaxdata:0xa0000000/dsa
 365     endif
 366     ifeq ($(OPENJDK_TARGET_CPU_BITS), 64)
 367       JAVAW_LDFLAGS  += -bpT:0x10000000000
 368     endif
 369     JAVAW_LDFLAGS  += -bM:UR
 370   endif # aix

via Work item https://jazz103.hursley.ibm.com:9443/jazz/web/projects/JTC-JAT#action=com.ibm.team.workitem.viewWorkItem&id=100052

[M-Davies]

I adapted the above work item for our needs, changing the parameter name to the more appropriate $1_LDFLAGS_SUFFIX

jdk/make/CompileLaunchers.gmk:105
  ifeq ($(OPENJDK_TARGET_OS), aix)
    ifeq ($(OPENJDK_TARGET_CPU_BITS), 32)
      $1_LDFLAGS_SUFFIX += -bmaxdata:0xa0000000/dsa
    else
      $1_LDFLAGS_SUFFIX += -bpT:0x10000000000
    endif
  endif

Unfixed Binary - MEMINFO Dump

Fixed Binary - MEMINFO Dump

To me, it doesn't appear to make that much difference. @zl-wang how are you checking where the text and data are loaded on AIX?

[zl-wang]

javacore only captures memory allocations known to the JVM. not all memory info at all.

procmap -X

checking the address range where object "java" is mapped, to see the difference.

[zl-wang]

For example:
milan:[9306292] procmap -X 9306292
9306292 : /usr/bin/bash

Start-ADD End-ADD SIZE MODE PSIZ TYPE VSID MAPPED OBJECT
0 10000000 262144K r-- m KERTXT 20002
100000000 1001439ef 1294K r-x s MAINTEXT a801e8 bash <<<. executable binary located
1100008b8 1100181d0 94K rw- sm MAINDATA 9b0e5b bash
1100181d0 1100882c0 448K rw- sm HEAP 9b0e5b
800200140000000 8002001400c4000 784K r-- sm KER/LDR 930f53
8fffffff0000000 900000000000000 262144K r-- s KER/LDR 980f18
900000000000680 900000000442de3 4361K r-x m SLIBTEXT 990019 /usr/lib/libc.a[shr_64.o]
900000000468400 900000000468f43 2K r-x m SLIBTEXT 990019 /usr/lib/libcrypt.a[shr_64.o]
900000000469000 9000000004692fb 0K r-x m SLIBTEXT 990019 /usr/lib/libdl.a[shr_64.o]
9000000004f8000 90000000053ab2e 266K r-x m SLIBTEXT 990019 /usr/lib/libpthreads.a[shr_xpg5_64.o]
900000000727000 90000000072d6f1 25K r-x m SLIBTEXT 990019 /usr/lib/security/GSA64
90000000072e280 9000000007663ca 224K r-x m SLIBTEXT 990019 /usr/lib/libiconv.a[shr4_64.o]
900000000767a40 900000000771aca 40K r-x m SLIBTEXT 990019 /usr/lib/libintl.a[libintl.so.1]
900000000ed9a00 900000000f2ff56 345K r-x m SLIBTEXT 990019 /usr/lib/libcurses.a[shr42_64.o]
9001000a0000a90 9001000a010cb40 1072K rw- sm PLIBDATA 890ec9 /usr/lib/libc.a[shr_64.o]
9001000a010da28 9001000a010dbb8 0K rw- sm PLIBDATA 890ec9 /usr/lib/libcrypt.a[shr_64.o]
9001000a0258000 9001000a02e701c 572K rw- sm PLIBDATA 890ec9 /usr/lib/libpthreads.a[shr_xpg5_64.o]
9001000a03ad098 9001000a0440768 589K rw- sm PLIBDATA 890ec9 /usr/lib/security/GSA64
9001000a0444480 9001000a045e4e8 104K rw- sm PLIBDATA 890ec9 /usr/lib/libiconv.a[shr4_64.o]
9001000a045f92e 9001000a04601d8 2K rw- sm PLIBDATA 890ec9 /usr/lib/libintl.a[libintl.so.1]
9001000a0477000 9001000a0477000 0K rw- sm PLIBDATA 890ec9 /usr/lib/libdl.a[shr_64.o]
9001000a06740e0 9001000a068c706 97K rw- sm PLIBDATA 890ec9 /usr/lib/libcurses.a[shr42_64.o]
900200140000000 900200150000000 262144K r-- s KER/LDR 9b001b
9ffffffd0000000 9ffffffe0000000 262144K r-- sm KER/LDR a000a
9ffffffe0000000 9fffffff0000000 262144K r-- sm KER/LDR e000e
9fffffff0000000 9fffffff000fa8e 62K r-x s SLIBTEXT b40174 /usr/ccs/bin/usla64
9fffffff000fa8e 9fffffff000fa8e 0K rw- s PLIBDATA b40174 /usr/ccs/bin/usla64
fffffff00000000 1000000000000000 4194304K rw- sm STACK 9f0f5f
Total 5515413K

[M-Davies]

Thanks @zl-wang

Before Above Fix:

100000000         100020961             130K r-x   s     MAINTEXT   886b77           java 
110000f5d         110001e30               3K rw-   sm    MAINDATA   873d68           java 

After Above Fix:

10000000000       100000209e9           130K r-x   s     MAINTEXT   8d969b           java 
10010000f7d       10010001e50             3K rw-   sm    MAINDATA   8e3d94           java 

Text/data is now loaded at 0x100,0000,0000 (1tb) as opposed to 0x1,0000,0000 (4gb). Will make the relevant PR now

[pshipton]

Fixed by the merges above.