When running with SCC enabled parts of the ROM class are in inaccessible memory. This causes crashes at runtime or when attempting to view romclass bytes in DDR.
To reproduce this issue one can use the following test application:
import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
public class StaticVarHandleTest {
static int field;
static int[] array = new int[20];
static final VarHandle FIELD;
static final VarHandle ARRAY;
static {
try {
FIELD = MethodHandles.lookup().in(StaticVarHandleTest.class).findStaticVarHandle(StaticVarHandleTest.class, "field", Integer.TYPE);
ARRAY = MethodHandles.arrayElementVarHandle(int[].class);
} catch(Exception e) {
throw new InternalError(e);
}
}
public static void main(String args[]) throws Exception {
int i = (int)FIELD.getVolatile();
System.out.println("=== ANSWER: field = " + i);
FIELD.getAndAdd(5);
System.out.println("=== ANSWER: field = " + field);
System.out.println("=== ANSWER: array = " + ARRAY.getAndAdd(array, 5, 5));
System.out.println("=== ANSWER: array = " + ARRAY.getAndAdd(array, 5, 5));
}
}
With a a openJDK MH enabled JDK, instructions are here https://github.com/eclipse/openj9/issues/11458#issue-762908255.
With SCC
~:~/scratch$ ./xa64150/jdk/bin/java StaticVarHandleTest.java
=== ANSWER: field = 0
Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F1348B386B0 Handler2=00007F134842CCC0 InaccessibleAddress=0000000000000008
RDI=00007F1349A7A6D0 RSI=0000000000000000 RAX=00000000000007F0 RBX=00007F1349A7A6A0
RCX=0000000000000000 RDX=000000000000007F R8=0000000000000001 R9=0000000000000004
R10=00000007054AF918 R11=0000000000000000 R12=0000000000107760 R13=00007F1348BED4C8
R14=0000000000000001 R15=00000000004E22E4
RIP=00007F1348AAFC97 GS=0000 FS=0000 RSP=00007F1349A7A330
EFlags=0000000000010206 CS=0033 RBP=0000000000019A00 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000008
xmm0 0000000000000005 (f: 5.000000, d: 2.470328e-323)
xmm1 00000007ffe5cde0 (f: 4293250560.000000, d: 1.697512e-313)
xmm2 00007f1349a7a6d0 (f: 1235724032.000000, d: 6.903126e-310)
xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm6 0000000000000019 (f: 25.000000, d: 1.235164e-322)
xmm7 00000007057b81f8 (f: 91980280.000000, d: 1.489941e-313)
xmm8 0000000000145650 (f: 1332816.000000, d: 6.584986e-318)
xmm9 00000000001456b0 (f: 1332912.000000, d: 6.585460e-318)
xmm10 00000000001456f0 (f: 1332976.000000, d: 6.585776e-318)
xmm11 0000000000145730 (f: 1333040.000000, d: 6.586093e-318)
xmm12 0000000000145790 (f: 1333136.000000, d: 6.586567e-318)
xmm13 2f6176616a4c5b3b (f: 1783388928.000000, d: 1.840923e-80)
xmm14 616a4c293b746365 (f: 997483392.000000, d: 1.848597e+161)
xmm15 432f656b6f766e69 (f: 1870032512.000000, d: 4.418618e+15)
Module=/home/tobi/scratch/xa64150/jdk/lib/compressedrefs/libj9vm29.so
Module_base_address=00007F1348A9B000
Target=2_90_20201211_000000 (Linux 4.4.0-197-generic)
CPU=amd64 (8 logical CPUs) (0x3eaeaf000 RAM)
----------- Stack Backtrace -----------
(0x00007F1348AAFC97 [libj9vm29.so+0x14c97])
(0x00007F1348AADB90 [libj9vm29.so+0x12b90])
(0x00007F1348B74972 [libj9vm29.so+0xd9972])
---------------------------------------
without it
~:~/scratch$ ./xa64150/jdk/bin/java -Xshareclasses:none StaticVarHandleTest.java
=== ANSWER: field = 0
=== ANSWER: field = 5
=== ANSWER: array = 0
=== ANSWER: array = 5
FYI @babsingh
@hangshao0 Can you please take a look at this.
This crash happens in cold run. May be related to the difference between putting the romclass(es) in SCC and outside of SCC.
The class that makes a difference is java/lang/invoke/VarHandleGuards.
If it is stored in the SCC, it crashes.
If it is not in the SCC, the crash goes away.
Why does storing VarHandleGuards in the SCC cause the crash?
More notes ...
The alternate solution is -Djava.lang.invoke.VarHandle.VAR_HANDLE_GUARDS=false, which prevents the usage of VarHandleGuards. VarHandleGuards contains precompiled bytecodes for VarHandle operations (or target methods). The usage of precompiled bytecodes is to avoid generating bytecodes at runtime.
Generally, the bytecodes are generated during runtime for OJDK MethodHandles/VarHandles. SCC works fine for the bytecodes generated during runtime.
fyi @DanHeidinga @fengxue-IS @gacholio @harryyu1994
This PR should also be tagged with project:MH.
Comparing the romclass of java/lang/invoke/VarHandleGuards stored outside of SCC vs stored in SCC, the only noticeable difference is the return bytecode.
genericReturn are changed to corresponding returnFromConstructor, return0, return1, return2, returnZ
For example dumping method java/lang/invoke/VarHandleGuards.guard__I()
When stored outside of SCC:
Name: guard__I
Signature: (Ljava/lang/invoke/VarHandle;Ljava/lang/invoke/VarHandle$AccessDescriptor;)I
Access Flags (30060018): default static final
Max Stack: 3
Thrown Exceptions (1):
java/lang/Throwable
Argument Count: 2
Temp Count: 1
0 aload0
1 invokevirtual 3 java/lang/invoke/VarHandle.isDirect()Z
4 ifeq 42
7 aload0getfield
8 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
11 getfield 6 java/lang/invoke/VarForm.methodType_table [Ljava/lang/invoke/MethodType;
14 aload1
15 getfield 8 java/lang/invoke/VarHandle$AccessDescriptor.type I
18 aaload
19 aload1
20 getfield 10 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeErased Ljava/lang/invoke/MethodType;
23 ifacmpne 42
26 aload0
27 aload0getfield
28 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
31 aload1
32 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
35 invokevirtual 12 java/lang/invoke/VarForm.getMemberName(I)Ljava/lang/invoke/MemberName;
38 invokestatic 78 java/lang/invoke/MethodHandle.linkToStatic(Ljava/lang/invoke/VarHandle;Ljava/lang/invoke/MemberName;)I
41 return1
42 aload0
43 aload1
44 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
47 invokevirtual 18 java/lang/invoke/VarHandle.getMethodHandle(I)Ljava/lang/invoke/MethodHandle;
50 astore2
51 aload2
52 aload1
53 getfield 19 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeInvoker Ljava/lang/invoke/MethodType;
56 invokevirtual 20 java/lang/invoke/MethodHandle.asType(Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;
59 aload0
60 invokevirtual 21 java/lang/invoke/VarHandle.asDirect()Ljava/lang/invoke/VarHandle;
63 invokevirtual 79 java/lang/invoke/MethodHandle.invokeBasic(Ljava/lang/invoke/VarHandle;)I
66 genericReturn
Debug Info:
Line Number Table (4):
Line: 393 PC: 0
Line: 394 PC: 26
Line: 397 PC: 42
Line: 398 PC: 51
Variables (3):
Slot: 0
Visibility Start: 0
Visibility End: 67
Visibility Length: 67
Name: handle
Signature: Ljava/lang/invoke/VarHandle;
Generic Signature: None
Slot: 1
Visibility Start: 0
Visibility End: 67
Visibility Length: 67
Name: ad
Signature: Ljava/lang/invoke/VarHandle$AccessDescriptor;
Generic Signature: None
Slot: 2
Visibility Start: 51
Visibility End: 67
Visibility Length: 16
Name: mh
Signature: Ljava/lang/invoke/MethodHandle;
Generic Signature: None
StackMapTable
Stackmaps(1):
pc: 42 same
When stored in SCC:
```
Name: guard__I
Signature: (Ljava/lang/invoke/VarHandle;Ljava/lang/invoke/VarHandle$AccessDescriptor;)I
Access Flags (30060018): default static final
Max Stack: 3
Thrown Exceptions (1):
java/lang/Throwable
Argument Count: 2
Temp Count: 1
0 aload0
1 invokevirtual 3 java/lang/invoke/VarHandle.isDirect()Z
4 ifeq 42
7 aload0getfield
8 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
11 getfield 6 java/lang/invoke/VarForm.methodType_table [Ljava/lang/invoke/MethodType;
14 aload1
15 getfield 8 java/lang/invoke/VarHandle$AccessDescriptor.type I
18 aaload
19 aload1
20 getfield 10 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeErased Ljava/lang/invoke/MethodType;
23 ifacmpne 42
26 aload0
27 aload0getfield
28 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
31 aload1
32 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
35 invokevirtual 12 java/lang/invoke/VarForm.getMemberName(I)Ljava/lang/invoke/MemberName;
38 invokestatic 78 java/lang/invoke/MethodHandle.linkToStatic(Ljava/lang/invoke/VarHandle;Ljava/lang/invoke/MemberName;)I
41 return1
42 aload0
43 aload1
44 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
47 invokevirtual 18 java/lang/invoke/VarHandle.getMethodHandle(I)Ljava/lang/invoke/MethodHandle;
50 astore2
51 aload2
52 aload1
53 getfield 19 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeInvoker Ljava/lang/invoke/MethodType;
56 invokevirtual 20 java/lang/invoke/MethodHandle.asType(Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;
59 aload0
60 invokevirtual 21 java/lang/invoke/VarHandle.asDirect()Ljava/lang/invoke/VarHandle;
63 invokevirtual 79 java/lang/invoke/MethodHandle.invokeBasic(Ljava/lang/invoke/VarHandle;)I
66 return1
Debug Info:
Line Number Table (4):
Line: 393 PC: 0
Line: 394 PC: 26
Line: 397 PC: 42
Line: 398 PC: 51
Variables (3):
Slot: 0
Visibility Start: 0
Visibility End: 67
Visibility Length: 67
Name: handle
Signature: Ljava/lang/invoke/VarHandle;
Generic Signature: None
Slot: 1
Visibility Start: 0
Visibility End: 67
Visibility Length: 67
Name: ad
Signature: Ljava/lang/invoke/VarHandle$AccessDescriptor;
Generic Signature: None
Slot: 2
Visibility Start: 51
Visibility End: 67
Visibility Length: 16
Name: mh
Signature: Ljava/lang/invoke/MethodHandle;
Generic Signature: None
StackMapTable
Stackmaps(1):
pc: 42 same
```
The only difference is 66 genericReturn changed to 66 return1.
The change of return bytecode is due to
https://github.com/eclipse/openj9/blob/master/runtime/bcutil/ROMClassBuilder.cpp#L652
which is expected. I don't see anything wrong with changing 66 genericReturn to 66 return1 here.
I can confirm the crash is caused by fixReturns() changing the return bytecode of the following method (114 genericReturn is changed to 114 return0) :
```
Name: guard_I_V
Signature: (Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/VarHandle$AccessDescriptor;)V
Access Flags (30060018): default static final
Max Stack: 4
Thrown Exceptions (1):
java/lang/Throwable
Argument Count: 3
Temp Count: 1
0 aload0
1 invokevirtual 3 java/lang/invoke/VarHandle.isDirect()Z
4 ifeq 45
7 aload0getfield
8 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
11 getfield 6 java/lang/invoke/VarForm.methodType_table [Ljava/lang/invoke/MethodType;
14 aload2
15 getfield 8 java/lang/invoke/VarHandle$AccessDescriptor.type I
18 aaload
19 aload2
20 getfield 10 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeErased Ljava/lang/invoke/MethodType;
23 ifacmpne 45
26 aload0
27 iload1
28 aload0getfield
29 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
32 aload2
33 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
36 invokevirtual 12 java/lang/invoke/VarForm.getMemberName(I)Ljava/lang/invoke/MemberName;
39 invokestatic 80 java/lang/invoke/MethodHandle.linkToStatic(Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/MemberName;)V
42 goto 114
45 aload0
46 invokevirtual 3 java/lang/invoke/VarHandle.isDirect()Z
49 ifeq 89
52 aload0getfield
53 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
56 aload2
57 getfield 8 java/lang/invoke/VarHandle$AccessDescriptor.type I
60 invokevirtual 24 java/lang/invoke/VarForm.getMethodType_V(I)Ljava/lang/invoke/MethodType;
63 aload2
64 getfield 10 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeErased Ljava/lang/invoke/MethodType;
67 ifacmpne 89
70 aload0
71 iload1
72 aload0getfield
73 getfield 5 java/lang/invoke/VarHandle.vform Ljava/lang/invoke/VarForm;
76 aload2
77 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
80 invokevirtual 12 java/lang/invoke/VarForm.getMemberName(I)Ljava/lang/invoke/MemberName;
83 invokestatic 80 java/lang/invoke/MethodHandle.linkToStatic(Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/MemberName;)V
86 goto 114
89 aload0
90 aload2
91 getfield 11 java/lang/invoke/VarHandle$AccessDescriptor.mode I
94 invokevirtual 18 java/lang/invoke/VarHandle.getMethodHandle(I)Ljava/lang/invoke/MethodHandle;
97 astore3
98 aload3
99 aload2
100 getfield 19 java/lang/invoke/VarHandle$AccessDescriptor.symbolicMethodTypeInvoker Ljava/lang/invoke/MethodType;
103 invokevirtual 20 java/lang/invoke/MethodHandle.asType(Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;
106 aload0
107 invokevirtual 21 java/lang/invoke/VarHandle.asDirect()Ljava/lang/invoke/VarHandle;
110 iload1
111 invokevirtual 81 java/lang/invoke/MethodHandle.invokeBasic(Ljava/lang/invoke/VarHandle;I)V
114 genericReturn
Debug Info:
Line Number Table (7):
Line: 405 PC: 0
Line: 406 PC: 26
Line: 408 PC: 45
Line: 409 PC: 70
Line: 412 PC: 89
Line: 413 PC: 98
Line: 415 PC: 114
Variables (4):
Slot: 0
Visibility Start: 0
Visibility End: 115
Visibility Length: 115
Name: handle
Signature: Ljava/lang/invoke/VarHandle;
Generic Signature: None
Slot: 1
Visibility Start: 0
Visibility End: 115
Visibility Length: 115
Name: arg0
Signature: I
Generic Signature: None
Slot: 2
Visibility Start: 0
Visibility End: 115
Visibility Length: 115
Name: ad
Signature: Ljava/lang/invoke/VarHandle$AccessDescriptor;
Generic Signature: None
Slot: 3
Visibility Start: 98
Visibility End: 114
Visibility Length: 16
Name: mh
Signature: Ljava/lang/invoke/MethodHandle;
Generic Signature: None
StackMapTable
Stackmaps(3):
pc: 45 same
pc: 89 same
pc: 114 same
```
fixReturns() is doing the following check on returnSlots before changing genericReturn to return0
https://github.com/eclipse/openj9/blob/46fa54c090e5891f35fe88255094c2ab14047fc3/runtime/stackmap/fixreturns.c#L309-L311
BytecodeInterpreter is doing the following check on returnSlots before change genericReturn at *_pc to return0
https://github.com/eclipse/openj9/blob/46fa54c090e5891f35fe88255094c2ab14047fc3/runtime/vm/BytecodeInterpreter.hpp#L7933-L7934
I guess these 2 checks are supposed to be equivalent. However, for the above method method guard_I_V(), it passed the check in fixReturns() with SCC on, but failed the check in BytecodeInterpreter with SCC off.
The return has been fixed in the SCC version, so how is genericreturn being invoked? Is the crash due to trying to write to R/O memory in the bytecode?
Is the crash due to trying to write to R/O memory in the bytecode?
No. It is not writing to a R/O memory in the bytecode. It crashed due to InaccessibleAddress=0000000000000008 (InaccessibleAddress could be different from run to run).
genericreturn is not modified when SCC is off. Even _pc is pointing to the return bytecode inside genericReturn(REGISTER_ARGS_LIST), but the following if check returns false, so we don't write to _pc. The test finished successfully.
if (returnSlots == (UDATA)(((UDATA*)frame) - _sp)) {
...
*_pc = (newBytecode + (U_8)returnSlots);
If I modify fixReturns() to skip fixing the return bytecode of guard_I_V() only, when SCC is on, I see _pc points to R/O memory in the SCC, but the above if check reruns false, we won't write to the R/O memory, The test finished successfully.
so how is genericreturn being invoked?
It won't crash if it is genericreturn, so I modified the VM to let it crash when running genericReturn(REGISTER_ARGS_LIST) of method guard_I_V()
The java stack is:
!stackslots 0x19a00
<19a00> *** BEGIN STACK WALK, flags = 00400001 walkThread = 0x0000000000019A00 ***
<19a00> ITERATE_O_SLOTS
<19a00> RECORD_BYTECODE_PC_OFFSET
<19a00> Initial values: walkSP = 0x00000000000D2078, PC = 0x0000000000000003, literals = 0x0000000000000000, A0 = 0x00000000000D20C8, j2iFrame = 0x0000000000000000, ELS = 0x00007F8CAE395AF0, decomp = 0x0000000000000000
<19a00> INL native method frame: bp = 0x00000000000D2098, sp = 0x00000000000D2078, pc = 0x0000000000000003, cp = 0x0000000000000000, arg0EA = 0x00000000000D20C8, flags = 0x0000000000000000
<19a00> Method: jdk/internal/misc/Unsafe.compareAndSetInt(Ljava/lang/Object;JII)Z !j9method 0x000000000002C228
<19a00> Using signature mapper
<19a00> Arguments starting at 0x00000000000D20C8 for UDATA(0x0000000000000006) slots
<19a00> O-Slot: a0[0x00000000000D20C8] = 0x0000000700000001
<19a00> O-Slot: a1[0x00000000000D20C0] = 0x0000000705488D38
<19a00> I-Slot: a2[0x00000000000D20B8] = 0x00007F8CA8163394
<19a00> I-Slot: a3[0x00000000000D20B0] = 0x0000000000000019
<19a00> I-Slot: a4[0x00000000000D20A8] = 0x0000000700000000
<19a00> I-Slot: a5[0x00000000000D20A0] = 0x0000000700000005
<19a00> Bytecode frame: bp = 0x00000000000D20E0, sp = 0x00000000000D20D0, pc = 0x00007F8CA81006CF, cp = 0x000000000002F2B0, arg0EA = 0x00000000000D2110, flags = 0x0000000000000000
<19a00> Method: jdk/internal/misc/Unsafe.weakCompareAndSetInt(Ljava/lang/Object;JII)Z !j9method 0x000000000002CF88
<19a00> Bytecode index = 7
<19a00> Using local mapper
<19a00> Locals starting at 0x00000000000D2110 for 0x0000000000000006 slots
<19a00> I-Slot: a0[0x00000000000D2110] = 0x0000000700000000
<19a00> I-Slot: a1[0x00000000000D2108] = 0x0000000705488D38
<19a00> I-Slot: a2[0x00000000000D2100] = 0x000000000015EA90
<19a00> I-Slot: a3[0x00000000000D20F8] = 0x0000000000000019
<19a00> I-Slot: a4[0x00000000000D20F0] = 0x0000000700000000
<19a00> I-Slot: a5[0x00000000000D20E8] = 0x0000000700000005
<19a00> Bytecode frame: bp = 0x00000000000D2128, sp = 0x00000000000D2118, pc = 0x00007F8CA8102DB2, cp = 0x000000000002F2B0, arg0EA = 0x00000000000D2158, flags = 0x0000000000000000
<19a00> Method: jdk/internal/misc/Unsafe.getAndAddInt(Ljava/lang/Object;JI)I !j9method 0x000000000002DC68
<19a00> Bytecode index = 18
<19a00> Using local mapper
<19a00> Locals starting at 0x00000000000D2158 for 0x0000000000000006 slots
<19a00> O-Slot: a0[0x00000000000D2158] = 0x0000000700000000
<19a00> O-Slot: a1[0x00000000000D2150] = 0x0000000705488D38
<19a00> I-Slot: a2[0x00000000000D2148] = 0x00000007FFE05F28
<19a00> I-Slot: a3[0x00000000000D2140] = 0x0000000000000019
<19a00> I-Slot: a4[0x00000000000D2138] = 0x0000000700000005
<19a00> I-Slot: t5[0x00000000000D2130] = 0x0000000700000000
<19a00> Bytecode frame: bp = 0x00000000000D2170, sp = 0x00000000000D2160, pc = 0x00007F8CA83AFE71, cp = 0x000000000015CF20, arg0EA = 0x00000000000D2188, flags = 0x0000000000000000
<19a00> Method: java/lang/invoke/VarHandleInts$FieldStaticReadWrite.getAndAdd(Ljava/lang/invoke/VarHandle;I)I !j9method 0x000000000015C9E0
<19a00> Bytecode index = 17
<19a00> Using local mapper
<19a00> Locals starting at 0x00000000000D2188 for 0x0000000000000003 slots
<19a00> I-Slot: a0[0x00000000000D2188] = 0x0000000700000000
<19a00> I-Slot: a1[0x00000000000D2180] = 0x0000000700000005
<19a00> I-Slot: t2[0x00000000000D2178] = 0x00000007FFE05F28
<19a00> Bytecode frame: bp = 0x00000000000D21A0, sp = 0x00000000000D2190, pc = 0x00007F8C6C0118FB, cp = 0x000000000015EB10, arg0EA = 0x00000000000D21C0, flags = 0x0000000000000000
<19a00> Method: java/lang/invoke/VarHandleGuards.guard_I_V(Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/VarHandle$AccessDescriptor;)V !j9method 0x0000000000160D60
<19a00> Bytecode index = 83
<19a00> Using local mapper
<19a00> Locals starting at 0x00000000000D21C0 for 0x0000000000000004 slots
<19a00> I-Slot: a0[0x00000000000D21C0] = 0x00000007FFE05F28
<19a00> I-Slot: a1[0x00000000000D21B8] = 0x0000000700000005
<19a00> I-Slot: a2[0x00000000000D21B0] = 0x00000007FFED4960
<19a00> I-Slot: t3[0x00000000000D21A8] = 0x00007F8CA83DB55F
<19a00> Bytecode frame: bp = 0x00000000000D21D8, sp = 0x00000000000D21C8, pc = 0x00007F8CA83DB55F, cp = 0x000000000014FD30, arg0EA = 0x00000000000D21E8, flags = 0x0000000000000000
<19a00> Method: StaticVarHandleTest.main([Ljava/lang/String;)V !j9method 0x00000000001501E0
<19a00> Bytecode index = 23
<19a00> Using local mapper
<19a00> Locals starting at 0x00000000000D21E8 for 0x0000000000000002 slots
<19a00> I-Slot: a0[0x00000000000D21E8] = 0x00000007FFF71C90
<19a00> I-Slot: t1[0x00000000000D21E0] = 0x0000000700000000
<19a00> JNI call-in frame: bp = 0x00000000000D2210, sp = 0x00000000000D21F0, pc = 0x00007F8CAD511454, cp = 0x0000000000000000, arg0EA = 0x00000000000D2210, flags = 0x0000000000020000
<19a00> New ELS = 0x0000000000000000
<19a00> JNI native method frame: bp = 0x00000000000D2298, sp = 0x00000000000D2218, pc = 0x0000000000000007, cp = 0x0000000000000000, arg0EA = 0x00000000000D2298, flags = 0x000000000000000C
<19a00> Object pushes starting at 0x00000000000D2218 for 12 slots
It says it is at bytecode index 23 of main().
<19a00> Method: StaticVarHandleTest.main([Ljava/lang/String;)V !j9method 0x00000000001501E0
<19a00> Bytecode index = 23
!bytecodes 0x00000000001501E0
Name: main
Signature: ([Ljava/lang/String;)V
Access Flags (8060009): public static
Max Stack: 5
Thrown Exceptions (1):
java/lang/Exception
Argument Count: 1
Temp Count: 1
0 getstatic 6 StaticVarHandleTest.FIELD Ljava/lang/invoke/VarHandle;
3 invokehandle 7 java/lang/invoke/VarHandle.getVolatile()I
6 istore1
7 getstatic 9 java/lang/System.out Ljava/io/PrintStream;
10 iload1
11 invokedynamic 0 bsm #0:makeConcatWithConstants(I)Ljava/lang/String;
16 nop
17 nop
18 invokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
21 getstatic 6 StaticVarHandleTest.FIELD Ljava/lang/invoke/VarHandle;
24 iconst5
25 invokehandle 13 java/lang/invoke/VarHandle.getAndAdd(I)V
...
The C stack is:
#13 VM_BytecodeInterpreterCompressed::genericReturn (_pc=<optimized out>, _sp=<optimized out>, this=<optimized out>) at BytecodeInterpreter.hpp:7961
#14 VM_BytecodeInterpreterCompressed::run (this=0x7f8cae395910, vmThread=0x2a) at BytecodeInterpreter.hpp:10597
#15 0x00007f8cad3c6a80 in bytecodeLoopCompressed (currentThread=<optimized out>) at BytecodeInterpreter.inc:110
#16 0x00007f8cad48dea2 in c_cInterpreter () at xcinterp.s:160
#17 0x00007f8cad439ea8 in runCallInMethod (env=0x7f8cae395a20, receiver=<optimized out>, clazz=0xd2230, methodID=0x7f8ca83fb5f8, args=0x7f8cae395dd8) at callin.cpp:1120
#18 0x00007f8cad454829 in gpProtectedRunCallInMethod (entryArg=0x7f8cae395d90) at jnicsup.cpp:259
#19 0x00007f8cac833663 in omrsig_protect (portLibrary=0x7f8cadb92340 <j9portLibrary>, fn=0x7f8cad497760 <signalProtectAndRunGlue>, fn_arg=0x7f8cae395d30,
handler=0x7f8cad451a60 <structuredSignalHandler>, handler_arg=0x19a00, flags=506, result=0x7f8cae395d28) at ../../omr/port/unix/omrsignal.c:425
#20 0x00007f8cad4977fc in gpProtectAndRun (function=0x7f8cad4547f0 <gpProtectedRunCallInMethod(void*)>, env=0x19a00, args=0x7f8cae395d90) at jniprotect.c:78
#21 0x00007f8cad456124 in gpCheckCallin (env=0x19a00, receiver=receiver@entry=0x0, cls=0xd2230, methodID=0x7f8ca83fb5f8, args=args@entry=0x7f8cae395dd8) at jnicsup.cpp:447
#22 0x00007f8cad4541fa in callStaticVoidMethod (env=<optimized out>, cls=<optimized out>, methodID=<optimized out>) at jnicgen.c:288
#23 0x00007f8caefb77e6 in JavaMain (_args=<optimized out>) at ./src/java.base/share/native/libjli/java.c:550
#24 0x00007f8caefba919 in ThreadJavaMain (args=<optimized out>) at ./src/java.base/unix/native/libjli/java_md.c:657
#25 0x00007f8cae5b46ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#26 0x00007f8caecef41d in clone () from /lib/x86_64-linux-gnu/libc.so.6
It crashed due to InaccessibleAddress=0000000000000008 (InaccessibleAddress could be different from run to run).
It does not always crash on InaccessibleAddress, sometimes it crashes on assertion failure here:
unimplementedBytecode(REGISTER_ARGS_LIST)
{
updateVMStruct(REGISTER_ARGS);
Assert_VM_unreachable();
There is no bytecode 23 in the above method, so that may explain why arbitrary things are happening. How is the PC being invalidly modified?
@babsingh How is the VarHandleGuards method being invoked? From what bytecode?
I don't see anywhere modifying the PC in invokehandle - maybe the ref is unresolved and the problem occurs in there, though I don't see how.
@hangshao0 Perhaps a debug build with some tracing in invokehandle would be in order.
Looks like it's the getAndAdd call at 25 that's got the wrong PC. The only place I see subtracting 2 from the PC has to do with dropToFrame and invokeinterface, which is not being used in this test.
@babsingh How is the VarHandleGuards method being invoked? From what bytecode?
invokehandle bytecode is called for VarHandle.getAndAdd(int)void. In invokehandle, the _sendMethod is set to VarHandleGuards.guard_I_V(VarHandle,int,AccessDescriptor)void/invokeStatic.
Inside invokehandle, I see the pc offset is 23, which is the same as what DDR shows here https://github.com/eclipse/openj9/issues/11582#issuecomment-758817994. I traced the _pc inside the previous iconstcall, I see pc offset is increased from 22 to 23 there.
Using javap to dump main(), iconst_5 is at index 22, and index 23 is invokevirtual
public static void main(java.lang.String[]) throws java.lang.Exception;
descriptor: ([Ljava/lang/String;)V
flags: (0x0009) ACC_PUBLIC, ACC_STATIC
Code:
stack=5, locals=2, args_size=1
0: getstatic #7 // Field FIELD:Ljava/lang/invoke/VarHandle;
3: invokevirtual #13 // Method java/lang/invoke/VarHandle.getVolatile:()I
6: istore_1
7: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
10: iload_1
11: invokedynamic #25, 0 // InvokeDynamic #0:makeConcatWithConstants:(I)Ljava/lang/String;
16: invokevirtual #29 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
19: getstatic #7 // Field FIELD:Ljava/lang/invoke/VarHandle;
22: iconst_5
23: invokevirtual #35 // Method java/lang/invoke/VarHandle.getAndAdd:(I)V
26: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
29: getstatic #39 // Field field:I
32: invokedynamic #25, 0 // InvokeDynamic #0:makeConcatWithConstants:(I)Ljava/lang/String;
37: invokevirtual #29 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
40: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
43: getstatic #43 // Field ARRAY:Ljava/lang/invoke/VarHandle;
46: getstatic #46 // Field array:[I
49: iconst_5
50: iconst_5
51: invokevirtual #50 // Method java/lang/invoke/VarHandle.getAndAdd:([III)Ljava/lang/Object;
54: invokedynamic #53, 0 // InvokeDynamic #1:makeConcatWithConstants:(Ljava/lang/Object;)Ljava/lang/String;
59: invokevirtual #29 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
62: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
65: getstatic #43 // Field ARRAY:Ljava/lang/invoke/VarHandle;
68: getstatic #46 // Field array:[I
71: iconst_5
72: iconst_5
73: invokevirtual #50 // Method java/lang/invoke/VarHandle.getAndAdd:([III)Ljava/lang/Object;
76: invokedynamic #53, 0 // InvokeDynamic #1:makeConcatWithConstants:(Ljava/lang/Object;)Ljava/lang/String;
81: invokevirtual #29 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
84: return
LineNumberTable:
line 17: 0
line 18: 7
line 19: 19
line 20: 26
line 21: 40
line 22: 62
line 23: 84
Exceptions:
throws java.lang.Exception
The bytecodes in the romclass and the indexes are a bit different:
11 invokedynamic 0 bsm #0:makeConcatWithConstants(I)Ljava/lang/String;
16 nop
17 nop
18 invokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
21 getstatic 6 StaticVarHandleTest.FIELD Ljava/lang/invoke/VarHandle;
24 iconst5
25 invokehandle 13 java/lang/invoke/VarHandle.getAndAdd(I)V
I wonder if the dumper isn't working right - invokedynamic is another 5-byte invoke, which we don't allow, so it gets rewritten in a 3-byte form followed by two NOPs.
We never change the bytecode indices of bytecodes coming in from class files, so I think the DDR dumper may be thinking that invokedynamic is 5 bytes in one place, and 3 bytes in another.
See what the output of cfdump says about the main method. We may be chasing a non-problem here with the bytecode indices.
cfdump shows:
cfdump -b StaticVarHandleTest.class
...
public static void main(java.lang.String[]) throws java.lang.Exception
0 JBgetstatic 7 StaticVarHandleTest.FIELD Ljava.lang.invoke.VarHandle;
3 JBinvokevirtual 13 java.lang.invoke.VarHandle.getVolatile()I
6 JBistore1
7 JBgetstatic 19 java.lang.System.out Ljava.io.PrintStream;
10 JBiload1
11 JBinvokedynamic
12 JBnop
13 JBaload 0
15 JBnop
16 JBinvokevirtual 29 java.io.PrintStream.println(Ljava.lang.String;)V
19 JBgetstatic 7 StaticVarHandleTest.FIELD Ljava.lang.invoke.VarHandle;
22 JBiconst5
23 JBinvokevirtual 35 java.lang.invoke.VarHandle.getAndAdd(I)V
26 JBgetstatic 19 java.lang.System.out Ljava.io.PrintStream;
29 JBgetstatic 39 StaticVarHandleTest.field I
32 JBinvokedynamic
33 JBnop
34 JBaload 0
36 JBnop
37 JBinvokevirtual 29 java.io.PrintStream.println(Ljava.lang.String;)V
40 JBgetstatic 19 java.lang.System.out Ljava.io.PrintStream;
43 JBgetstatic 43 StaticVarHandleTest.ARRAY Ljava.lang.invoke.VarHandle;
46 JBgetstatic 46 StaticVarHandleTest.array [I
49 JBiconst5
50 JBiconst5
51 JBinvokevirtual 50 java.lang.invoke.VarHandle.getAndAdd([III)Ljava.lang.Object;
54 JBinvokedynamic
55 JBnop
56 JBsaload
57 JBnop
58 JBnop
59 JBinvokevirtual 29 java.io.PrintStream.println(Ljava.lang.String;)V
62 JBgetstatic 19 java.lang.System.out Ljava.io.PrintStream;
65 JBgetstatic 43 StaticVarHandleTest.ARRAY Ljava.lang.invoke.VarHandle;
68 JBgetstatic 46 StaticVarHandleTest.array [I
71 JBiconst5
72 JBiconst5
73 JBinvokevirtual 50 java.lang.invoke.VarHandle.getAndAdd([III)Ljava.lang.Object;
76 JBinvokedynamic
77 JBnop
78 JBsaload
79 JBnop
80 JBnop
81 JBinvokevirtual 29 java.io.PrintStream.println(Ljava.lang.String;)V
84 JBreturn
So invokedynamic isn't printing properly there at all. I wonder if -x instead of -b works better.
cfdump -x StaticVarHandleTest.class
Name: main
Signature: ([Ljava/lang/String;)V
Access Flags (8060009): (hasExceptionInfo) (has debug info) (has method handle invokes)
access: public static
Extended modfiers (0):
Max Stack: 5
Thrown Exceptions (1):
java/lang/Exception
Argument Count: 1
Temp Count: 1
0 JBgetstatic 6 StaticVarHandleTest.FIELD Ljava/lang/invoke/VarHandle;
3 JBinvokehandle 7 java/lang/invoke/VarHandle.getVolatile()I
6 JBistore1
7 JBgetstatic 9 java/lang/System.out Ljava/io/PrintStream;
10 JBiload1
11 JBinvokedynamic 0 makeConcatWithConstants(I)Ljava/lang/String; bsm 0 java/lang/invoke/StringConcatFactory.makeConcatWithConstants(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
14 JBnop
15 JBnop
16 JBinvokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
19 JBgetstatic 6 StaticVarHandleTest.FIELD Ljava/lang/invoke/VarHandle;
22 JBiconst5
23 JBinvokehandle 13 java/lang/invoke/VarHandle.getAndAdd(I)V
26 JBgetstatic 9 java/lang/System.out Ljava/io/PrintStream;
29 JBgetstatic 14 StaticVarHandleTest.field I
32 JBinvokedynamic 1 makeConcatWithConstants(I)Ljava/lang/String; bsm 0 java/lang/invoke/StringConcatFactory.makeConcatWithConstants(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
35 JBnop
36 JBnop
37 JBinvokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
40 JBgetstatic 9 java/lang/System.out Ljava/io/PrintStream;
43 JBgetstatic 15 StaticVarHandleTest.ARRAY Ljava/lang/invoke/VarHandle;
46 JBgetstatic 16 StaticVarHandleTest.array [I
49 JBiconst5
50 JBiconst5
51 JBinvokehandle 17 java/lang/invoke/VarHandle.getAndAdd([III)Ljava/lang/Object;
54 JBinvokedynamic 2 makeConcatWithConstants(Ljava/lang/Object;)Ljava/lang/String; bsm 1 java/lang/invoke/StringConcatFactory.makeConcatWithConstants(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
57 JBnop
58 JBnop
59 JBinvokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
62 JBgetstatic 9 java/lang/System.out Ljava/io/PrintStream;
65 JBgetstatic 15 StaticVarHandleTest.ARRAY Ljava/lang/invoke/VarHandle;
68 JBgetstatic 16 StaticVarHandleTest.array [I
71 JBiconst5
72 JBiconst5
73 JBinvokehandle 17 java/lang/invoke/VarHandle.getAndAdd([III)Ljava/lang/Object;
76 JBinvokedynamic 3 makeConcatWithConstants(Ljava/lang/Object;)Ljava/lang/String; bsm 1 java/lang/invoke/StringConcatFactory.makeConcatWithConstants(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
79 JBnop
80 JBnop
81 JBinvokevirtual 11 java/io/PrintStream.println(Ljava/lang/String;)V
84 JBgenericReturn
Debug Info:
Line Number Table (7):
Line: 17 PC: 0
Line: 18 PC: 7
Line: 19 PC: 19
Line: 20 PC: 26
Line: 21 PC: 40
Line: 22 PC: 62
Line: 23 PC: 84
Variables (0):
Yes, that output is correct, so DDR needs to be fixed. @keithc-ca
This means the wonky bytecode index has nothing to do with the issue being investigated here.
OK, I know what is going on here.
In the bytecode of java/lang/invoke/VarHandleGuards.guard_I_V(), it tells us it will link to a method that has return type void:
83 invokestatic 80 java/lang/invoke/MethodHandle.linkToStatic(Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/MemberName;)V
fixReturns() changed the return bytecode based on the void return.
However at runtime from the core file, the method that gets linked is getAndAdd(Ljava/lang/invoke/VarHandle;I)I, which has a return type int. This inconsistency invalidates the fixed return bytecode, causing the crash.
<19a00> Method: java/lang/invoke/VarHandleInts$FieldStaticReadWrite.getAndAdd(Ljava/lang/invoke/VarHandle;I)I
...
...
<19a00> Method: java/lang/invoke/VarHandleGuards.guard_I_V(Ljava/lang/invoke/VarHandle;ILjava/lang/invoke/VarHandle$AccessDescriptor;)V !j9method 0x0000000000160D60
<19a00> Bytecode index = 83
I think even if we need to link to a method whose type is (Ljava/lang/invoke/VarHandle;I)I, it should be linked through guard_I_I() rather than guard_I_V().
I added some printf to the input parameters of guard_I_V(), I can see the methodType is inconsistent.
ad.symbolicMethodTypeInvoker is (VarHandle,int)void
handle.vform.getMemberName(ad.mode).getMethodType() is (VarHandle,int)int
There is an OpenJDK bug on the inconsistent methodType issue: https://bugs.openjdk.java.net/browse/JDK-8255375.
I see in the latest OpenJDK code, a check for methodType is already added to all the methods in VarHandleGuards. The check is performed if VarHandle.hasInvokeExactBehavior() is true. However, the build that crashes here does not have that change yet.
If by default VarHandle.hasInvokeExactBehavior() is not true, we may need to patch OpenJDK to make it true. Or not going through VarHandleGuards methods when there is a methodType mismatch.
Good find! There's nothing practical we could fix here, so I suppose we just wait to integrate the OpenJDK fix.
@babsingh tried the latest OpenJDK code, he found that VarHandle.hasInvokeExactBehavior() is false by default. So the method type checks in java/lang/invoke/VarHandleGuards are not turned on by default.
we may need to patch OpenJDK to make it true. Or not going through VarHandleGuards methods when there is a methodType mismatch.
If we change hasInvokeExactBehavior() to be true by default, it will throw a WrongMethodTypeException. It is different from the default behaviour of RI.
We may patch OpenJDK code to find a way to skip VarHandleGuards methods, similar to -Djava.lang.invoke.VarHandle.VAR_HANDLE_GUARDS=false, if there is a methodType mismatch.
Alternatively, we can just change on the openj9 side to not put java/lang/invoke/VarHandleGuards in the SCC, as it only happens to classes that are put into SCC that we fixed return bytecode beforehand.
hasInvokeExactBehavior() is not used in JDK11. So, JDK11 OpenJDK patch != JDK16 OpenJDK patch. We don't have to worry about JDK8 since it does not have VarHandles.-Djava.lang.invoke.VarHandle.VAR_HANDLE_GUARDS=false will completely disable the optimization, which utilizes precompiled bytecodes instead of generating bytecodes during runtime.VarHandleGuards in the SCC, we can keep utilizing the VAR_HANDLE_GUARDS optimization. This solution will also work with all extension repos (JDK8/11/16). So, I vote for not storing VarHandleGuards in the SCC.as it only happens to classes that are put into SCC that we fixed return bytecode beforehand.
I realize BytecodeInterpreter is also fixing the return bytecode here:
https://github.com/eclipse/openj9/blob/46fa54c090e5891f35fe88255094c2ab14047fc3/runtime/vm/BytecodeInterpreter.hpp#L7949
If we enter VarHandleGuards.guard_I_V() twice. The first time it linked to an exact (VarHandle,int)void, the above line will fix the return byecode to return0. The second time, if it linked to (VarHandle,int)int,return0 is not correct anymore, then the VM will crash.
I can crash the VM with the following test case even with -Xshareclasses:none
import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
public class StaticVarHandleTest_v1 {
static int field;
static int[] array = new int[20];
static final VarHandle FIELD;
static final VarHandle ARRAY;
static {
try {
FIELD = MethodHandles.lookup().in(StaticVarHandleTest.class).findStaticVarHandle(StaticVarHandleTest.class, "field", Integer.TYPE);
ARRAY = MethodHandles.arrayElementVarHandle(int[].class);
} catch(Exception e) {
throw new InternalError(e);
}
}
public static void main(String args[]) throws Exception {
int i = (int)FIELD.getVolatile();
System.out.println("=== ANSWER: field = " + i);
FIELD.setVolatile(2);
FIELD.getAndAdd(5);
System.out.println("=== ANSWER: field = " + field);
System.out.println("=== ANSWER: array = " + ARRAY.getAndAdd(array, 5, 5));
System.out.println("=== ANSWER: array = " + ARRAY.getAndAdd(array, 5, 5));
}
}
java -Xint -Xshareclasses:none StaticVarHandleTest_v1
=== ANSWER: field = 0
Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F2D6F4BA6B0 Handler2=00007F2D6EDAECC0 InaccessibleAddress=000000070000000
If the method return type does not match the signature we invoked, the VM will go awry, and there's nothing to be done about that (in particular, think of a method expecting an object to be returned, but the called method actually returns anything else - we'd potentially treat random stack data as an object).
Those VarHandleGuards methods are basically methods whose byecode can vary at runtime.
The guard method is set in MethodHandleNatives.varHandleOperationLinkerMethod, which is invoked during resolution of VarHandles. Guess: the guardReturnType is being set incorrectly. If we correct the guardReturnType at this point, @hangshao0 will this fix the issue?
Guess: the guardReturnType is being set incorrectly. If we correct the guardReturnType at this point, @hangshao0 will this fix the issue?
The guardReturnType is void there. Things still seem to be correct in varHandleOperationLinkerMethod(). The linker guard_I_V() here will eventually be passed back through MethodHandleNatives.linkMethod() to the caller. It is the caller that incorrectly uses linker guard_I_V() to link to (VarHandle,int)int, which causes problems.
We may not be able to know if there is a methodtype mismatch (between the linker and linked method) until using VarHandleGuards linker to link to a method. At that time, it may be too late to switch to a non-VarHandleGuards linker.
Probably we need to choose to disable VarHandleGuards.
Probably we need to choose to disable
VarHandleGuards.
I agree. Temporarily, we can use -Djava.lang.invoke.VarHandle.VAR_HANDLE_GUARDS=false to completely disable the optimization.
In the future, I or Jack will look into avoiding the methodtype mismatch during the call to MethodHandleNatives.linkMethod, which happens from MethodHandleResolver.linkCallerMethod. fyi @fengxue-IS .
@hangshao0 Thanks for triaging and highlighting the key issues.
At this point, the title of this issue:
Issue with hidden classes when SCC is enabled
is not correct anymore. VarHandleGuards is not a hidden class and the problem is there either with or without SCC.
Probably something like "MethodType mismatch of VarHandleGuards linker methods crash the VM" better describes this issue.