Graal: Reflection configuration file generator tool

[serkan-ozal]

Hi,

I have just started playing with GraalVM (especially for native image build feature) and thought that automated reflection configuration file generator tool from given jar file might be useful.

I am thinking of implementing a tool which parses bytecode of all classes in the given jar, introspecting reflective calls (field/method accesses though reflection, etc ...) and adding their conf to the generated file.

I know that this approach doesn't/can't cover all cases (for example dynamic class loading using Class.forName() is not available) but I think it might be good start point for later possible improvements until reflection is fully supported by GraalVM later

Does it make sense or am I missing something?

[cstancu]

@serkan-ozal that's a good idea, however I think this goal can be achieved easier inside SubstrateVM, during image building, not in a separate tool. Graal provides a plugin mechanism that enables you to intercept calls during bytecode parsing. Then you can either substitute the intercepted calls with your own logic or just trigger other side effects. See here for example how we intercept calls to java.lang.reflect.Proxy API and automatically generate proxy classes during image building. Calls to the reflection API could be intercepted in the same way and, as long as their arguments are constant, the referred elements (fields, methods, classes) can be registered for reflection. Using the invocation plugins you can take this one step further and not only register the elements for reflection, you can actually replace the reflective call with the actual element being accessed. Then Graal constant folding can take it from there and improve your code further. We are already working on this feature, and it should make reflection easier to work with. In situation where the arguments are not constant and you can not unequivocally determine which element is being accessed, there is not much you can do, and I don't think that another tool could do either.

[serkan-ozal]

Hi @cstancu
Agreed that it is better to implement in Graal rather that in 3rd party tool. I was not aware of that it is on your roadmap in short term.

Is there any issue or branch for this feature that I can track the changes and status?

[cstancu]

We don't have an issue or branch on github for this work. I will notify you here when the commit is merged.

[serkan-ozal]

Great! any ETA?

[cstancu]

No concrete ETA, but not more than few weeks. The changes also include some improvements in our sun.misc.Unsafe automatic offset recomputation which is tightly coupled with reflection.

[serkan-ozal]

Hi @cstancu

As you said, the implementation requires constant arguments and dynamic arguments will not be taken into consideration.

Therefore, I am thinking of implementing a tool which will intercept reflection calls (lets include Class.forName(...) and ClassLoader.loadClass(...) also) and will collect informations at runtime about which classes/methods/fields are referenced. I know it is not easy but feasible.

Note that this tool is not used at compile time but at runtime while running regular Java application (not from native image). It helps to discover the classes/methods/fields used by reflection with both of constant or dynamic arguments. In fact the collected information can be considered as profile data. Then the profile data can be used for generating required reflection configurations to GraalVM while building native image.

What do you think?

[cstancu]

@serkan-ozal I think such a solution is feasible using bytecode instrumentation. You can instrument the reflective calls and dump the concrete values of the arguments. The downside of this approach is that it is workload dependent, its success depends on how general your input data is and what percentage of the code is executed. In terms of code reachability a runtime profiling solution is at the other end of the spectrum when compared to a static analysis. The static analysis needs to be conservative and over-approximate the set of reachable classes/fields/methods. However, profiling can automate the discovery of some of the reflectively accessed elements, especially of those specified dynamically that the static analysis wouldn't be able to detect. The two approaches are not exclusive, you could use such a tool to augment the set of reflectively accessed elements discovered via static analysis.

[serkan-ozal]

@cstancu
I have just started this project https://github.com/serkan-ozal/sarras for implementing approaches discussed below.
Lets see whether it will be helpful :)

[cstancu]

@serkan-ozal https://github.com/oracle/graal/commit/3c85875fd8990b61b0365a83823449e4038f12eb improves native-image reflection support by automatically detecting reflective calls and intrinsifying them to the corresponding target elements statically. Please read the updated REFLECTION.md for details.

[peter-hofer]

We now have tooling for this, please refer to the documentation for details.

[sureshg]

This is nice. Haven't seen about the native image configure in release notes for rc13.

[cstancu]

@sureshg it was merged after the rc13 release. It will be part of rc14.