Graal: Sharing precompiled Source objects between contexts

[ralleman]

This has been touched a bit in this issue https://github.com/graalvm/graaljs/issues/67, but my use case is a bit different.

I'm writing a service that runs untrusted code. The service allows users to require or load other sources, but there are many JavaScript libraries (or libraries in other languages supported by Graal) that get reused over and over again in those pieces of untrusted code, e.g. lodash, underscore, moment, etc. At the moment, I can create and cache a Source object for each of these libraries, but when I attach it to a new context, it needs to be recompiled, so there's not much saving in the caching of the Source. That's where the other issue above got to. In my case, I want to create a new context for every piece of untrusted code that gets run, so I don't want to bind a context to a thread.

So I guess this is an enhancement request in which Source objects remain precompiled and reusable in multiple contexts within multiple threads. IMO, it is better to live with the overhead of the pieces of each Source being JIT compiled as needed synchronously, than to have to completely recompile the Source over and over for each context.

[wirthi]

Hi,

if you follow the description in http://www.graalvm.org/docs/graalvm-as-a-platform/embed/#enable-source-caching then source caching should be enabled. Whether the different Contexts run in the same or in different threads should not make much of a difference (provided the affected languages don't fall into slow paths or a race conditions because of multi-threading).

Can you please clarify how "your case is different" - did you actually try to share one Engine between several Contexts and noted performance does not go up, or do you get any errors?

Thanks,
Christian

[ralleman]

Hi Christian,

I haven't tested this. I was specifically picking up on this post https://github.com/graalvm/graaljs/issues/67 where other Christian (@chumer) states:

Currently for Graal.JS the ASTs and code cache is only shared if the previous context was closed.

My situation is different in that I'm creating a single Source object per source file and sharing it globally within the entire JVM, including all threads and all Contexts (using a single Engine). If the Source is getting recompiled because it's currently in use in another thread's open Context, then this would seem wasteful and detrimental to performance. Logically the Source object should be getting more and more JIT compiled as time goes on and get shared between all threads and all Contexts irrespective of whether they're closed or open.

Of course that comment was a while back, so this may have all changed by now.

Richard

[wirthi]

Hi,

the point is: you need to open and close Contexts (that all share one Engine) for source caching to work. If you share an identical Source object between those, it will optimize and eventually get fast.

This is unaffected by threading, with the exception that in a multi-threaded setup, if you start many threads (contexts) on the source object concurrently, none of them might profit from the compiled source in their first run(s), as all of them are starting with the still non-optimized one. Even worse, all those threads compete with Graal's compiler threads. So avoid starting "all at once" if you need good performance early on.

If the Source is getting recompiled because it's currently in use in another thread's open Context ...

That should not be the case in general - but of course might happen depending on the code.

-- Christian

[chumer]

Note that we have plans to allow multi-threaded sharing of ASTs for JavaScript. It is not done yet.