Openj9: Distinguish user arguments from generated arguments

[theresa-m]

Currently it cannot be distinguished whether arguments in J9VMInitArgs are created by the vm or by the user. JVM_GetVmArguments is expected to return only the user generated arguments. JVM_GetVmArguments should be updated to reflect this when the information distinguishing user args or generated args is available. @pdbain-ibm is working on making this information available.

[pdbain-ibm]

This is also related to https://github.com/eclipse/openj9/issues/2377 and https://github.com/eclipse/openj9/issues/2378.
Currently the list of VM arguments reported by the VM may be problematic if they are used to launch another VM since they contain JVM and process-specific information.
@theresa-m could you please provide examples of the arguments provided by OpenJ9 and the reference implementation?
Thanks.

[theresa-m]

My command line:
java --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED -Xint TestProgram

Hotspot output from getVmArguments:

--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED
-Xint

Current OpenJ9 output:
-Xoptionsfile=/team/theresa-m/builds/11_getvmargs/jdk/lib/options.default -Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple -Xjcl:jclse11_29 -Dcom.ibm.oti.vm.bootstrap.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib -Dsun.boot.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib -Djava.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib:/usr/local/cuda-5.5/lib64:.:/usr/lib64:/usr/lib -Djava.home=/team/theresa-m/builds/11_getvmargs/jdk -Duser.dir=/team/theresa-m/builds/11_getvmargs --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED -Xint -Djava.class.path=. -Dsun.java.command=Main -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher.pid=12068

[DanHeidinga]

Does Hotspot have options files? Can we confirm whether the options in those files (assuming they exist) are also reported by this api?

[pshipton]

@argument files one or more argument files containing options

[theresa-m]

confirmed arguments in option files (hotspot argument files) are also reported
$ oracle-build/jdk-11/bin/java @options -Xint TestProgram

options file content:
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED

output from program calling getRuntimeArguments():

--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED
-Xint

[DanHeidinga]

As a first cut of the API, I would suggest to return all the options and see if that is sufficient to unblock the test issues.

We can look at refining the support and limiting the options returned once we've got the initial implementation in place

[pdbain-ibm]

I suspect the @options files are processed by the launcher. In that case, OpenJ9 builds should support that as well. @theresa-m could you please test? I suggest also running with -verbose:init to see what we get from the launcher.

[theresa-m]

OpenJ9 includes options arguments as well.
bin/java @options -Xint -verbose:init TestProgram
Output:

-Xoptionsfile=/team/theresa-m/builds/11_getvmargs/jdk/lib/options.default
-Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple
-Xjcl:jclse11_29
-Dcom.ibm.oti.vm.bootstrap.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib
-Dsun.boot.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib
-Djava.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib:/usr/local/cuda-5.5/lib64:.:/usr/lib64:/usr/lib
-Djava.home=/team/theresa-m/builds/11_getvmargs/jdk
-Duser.dir=/team/theresa-m/builds/11_getvmargs
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED
-Xint
-verbose:init
-Djava.class.path=.
-Dsun.java.command=TestProgram
-Dsun.java.launcher=SUN_STANDARD
-Dsun.java.launcher.pid=9899

-verbose:init output adds the same argument list:

Adding command line arguments
Option 0 optionString="-Xoptionsfile=/team/theresa-m/builds/11_getvmargs/jdk/lib/options.default" extraInfo=(nil) from environment variable ="N/A"
Option 1 optionString="-Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple" extraInfo=(nil) from environment variable ="N/A"
Option 2 optionString="-Xjcl:jclse11_29" extraInfo=(nil) from environment variable ="N/A"
Option 3 optionString="-Dcom.ibm.oti.vm.bootstrap.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib" extraInfo=(nil) from environment variable ="N/A"
Option 4 optionString="-Dsun.boot.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib" extraInfo=(nil) from environment variable ="N/A"
Option 5 optionString="-Djava.library.path=/team/theresa-m/builds/11_getvmargs/jdk/lib/compressedrefs:/team/theresa-m/builds/11_getvmargs/jdk/lib:/usr/local/cuda-5.5/lib64:.:/usr/lib64:/usr/lib" extraInfo=(nil) from environment variable ="N/A"
Option 6 optionString="-Djava.home=/team/theresa-m/builds/11_getvmargs/jdk" extraInfo=(nil) from environment variable ="N/A"
Option 7 optionString="-Duser.dir=/team/theresa-m/builds/11_getvmargs" extraInfo=(nil) from environment variable ="N/A"
Option 8 optionString="--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED" extraInfo=(nil) from environment variable ="N/A"
Option 9 optionString="-Xint" extraInfo=(nil) from environment variable ="N/A"
Option 10 optionString="-verbose:init" extraInfo=(nil) from environment variable ="N/A"
Option 11 optionString="-Djava.class.path=." extraInfo=(nil) from environment variable ="N/A"
Option 12 optionString="-Dsun.java.command=TestProgram" extraInfo=(nil) from environment variable ="N/A"
Option 13 optionString="-Dsun.java.launcher=SUN_STANDARD" extraInfo=(nil) from environment variable ="N/A"
Option 14 optionString="-Dsun.java.launcher.pid=9899" extraInfo=(nil) from environment variable ="N/A"

[AidanHa]

Hello, I will be working on this issue.

At the moment, my understanding is that we want a way to distinguish arguments created by the user and arguments generated by the VM. I was thinking of simply making a new flag for J9VMInitArgs that is set if the argument is user raised.

I checked the options file for OpenJ9 and it seems like the only option present is:
Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple
From what I understand above, the -Xint option should be printing options from the option file if the argument @options exist (which is exactly what OpenJ9 is doing).

As a first cut of the API, I would suggest to return all the options and see if that is sufficient to unblock the test issues.

We can look at refining the support and limiting the options returned once we've got the initial implementation in place

I didn't quite understand this part @DanHeidinga. Could you help me by clarifying what you meant here?

[theresa-m]

Hey @AidanHa the "TestProgram" that was used to get the example output was actually just calling the Java method jdk.internal.misc.VM.getRuntimeArguments() and printing the output. -Xint and the user @ option file were just examples of user arguments but they could be any valid arguments. Right now the getRuntimeArguments method is returning more information than needed in our implementation because we can't distinguish between the user and vm arguments. This is the first cut Dan is referring to. Ideally with your changes this will be able to return the correct output.

[AidanHa]

Thank you for the clarification! Would it be possible to send me the "TestProgram" just so I can accurately reproduce the issue?

So to clarify I will initially be making changes to getRuntimeArguments so that it ONLY returns user arguments.

[pdbain-ibm]

@AidanHa Please include me in this discussion, as this was my implementation.
vmargs constructs the argument list from a number of sources, some provided by the user and some generated automatically, such as the bootstrap classpath extensions directory, etc.

There are other arguments such as -Xoptionsfile which are expanded to the contents of said file: we should include the latter and discard the -Xoptionsfile option. We should ignore options.default though, as it is not user-supplied.

[pdbain-ibm]

Thank you for the clarification! Would it be possible to send me the "TestProgram" just so I can accurately reproduce the issue?

So to clarify I will initially be making changes to getRuntimeArguments so that it ONLY returns user arguments.

You can look at the changes for jps:
https://github.com/eclipse/openj9/pull/4650/files#diff-e1b525f5e520f5d1caa62b5fd1205925

[theresa-m]

It doesn't look like I kept my TestProgram but let me know if you need any help making one of your own when you get to that point.

Start first with your solution to distinguish between user and vm arguments. The resources Peter linked will be a good place to start looking.

[AidanHa]

Hello, The following options seem to be part of a list of "user arguments" in addLauncherArgs()
-Djava.class.path=.
-Dsun.java.command=Test
-Dsun.java.launcher=SUN_STANDARD
-Dsun.java.launcher.pid=75488

Should they be returned in the JVM_GetVmArguments() as well? Or would I treat them as vm arguments

[pshipton]

See https://github.com/eclipse/openj9/issues/2974#issuecomment-423609064
We are trying to match the Hotspot behavior

[AidanHa]

Regarding this issue, I think we should still keep the old implementation of JVM_GetVmArguments() somewhere. If we are planning on keeping the old implementation, there are 2 options we can take:

1: Modify the current JVM_GetVmArguments() to return user arguments (just like Hotspot), and create a new function (EG: JVM_GetALLVmArguments()) that returns all arguments.

2: Keep the current implementation of JVM_GetVmArguments() and add a new function, JVM_GetUserVmArguments() that returns user arguments only.

Thoughts?

[pdbain-ibm]

Assuming it doesn't break us (too much), I favour option 1 if it makes the result of JVM_GetVmArguments() more congruent with the reference implementation.

[theresa-m]

this issue was resolved