Graaljs: Exception when run from uber-jar

[virtualzero2]

I am trying to use Graal.JS as a Nashorn replacement. I basically followed the instructions here. I think the one relevant caveat is that I running this on Windows 10.

The js and js-scriptengine artifacts are brought in using Maven and the project runs fine using IntelliJ. I use Maven to create an uber-jar and when I try and run a script from this uber-jar I get the following exception:

Mar 4, 2019 @ 3:36:57 PM - EXCEPTION - org.graalvm.polyglot.PolyglotException: java.lang.IllegalStateException: No language for id regex found. Supported languages are: [js]
    at com.oracle.truffle.polyglot.PolyglotEngineImpl.findLanguage(PolyglotEngineImpl.java:446)
    at com.oracle.truffle.polyglot.PolyglotImpl$EngineImpl.parseForLanguage(PolyglotImpl.java:473)
    at com.oracle.truffle.api.TruffleLanguage$Env.parse(TruffleLanguage.java:1786)
    at com.oracle.truffle.js.runtime.JSContext.getRegexEngine(JSContext.java:958)
    at com.oracle.truffle.js.runtime.RegexCompilerInterface.compile(RegexCompilerInterface.java:75)
    at com.oracle.truffle.js.runtime.RegexCompilerInterface.compile(RegexCompilerInterface.java:66)
    at com.oracle.truffle.js.nodes.access.RegExpLiteralNode.execute(RegExpLiteralNode.java:92)
    at com.oracle.truffle.js.nodes.function.JSFunctionCallNode$InvokeNNode.executeFillObjectArray(JSFunctionCallNode.java:898)
    at com.oracle.truffle.js.nodes.function.JSFunctionCallNode$InvokeNNode.createArguments(JSFunctionCallNode.java:891)
    at com.oracle.truffle.js.nodes.function.JSFunctionCallNode$InvokeNode.execute(JSFunctionCallNode.java:754)
    at com.oracle.truffle.js.nodes.access.JSWriteCurrentFrameSlotNodeGen.execute_generic3(JSWriteCurrentFrameSlotNodeGen.java:149)
    at com.oracle.truffle.js.nodes.access.JSWriteCurrentFrameSlotNodeGen.execute(JSWriteCurrentFrameSlotNodeGen.java:84)
    at com.oracle.truffle.js.nodes.access.JSWriteCurrentFrameSlotNodeGen.executeVoid(JSWriteCurrentFrameSlotNodeGen.java:281)
    at com.oracle.truffle.js.nodes.control.AbstractBlockNode.executeVoid(AbstractBlockNode.java:74)
    at com.oracle.truffle.js.nodes.control.BlockNode.execute(BlockNode.java:62)
    at com.oracle.truffle.js.nodes.function.FunctionBodyNode.execute(FunctionBodyNode.java:66)
    at com.oracle.truffle.js.nodes.function.FunctionRootNode.executeInRealm(FunctionRootNode.java:147)
    at com.oracle.truffle.js.runtime.JavaScriptRealmBoundaryRootNode.execute(JavaScriptRealmBoundaryRootNode.java:92)

I assume the uber-jar is messing up the manifests or something but it is out of my skill range to be sure.

Any thoughts on this?

[wirthi]

Hi @virtualzero2,

when you do mvn package from a clean state (and have your local maven repository clean) you should see a download of a regex-*.jar`, e.g.:

Downloaded: https://repo.maven.apache.org/maven2/org/graalvm/regex/regex/1.0.0-rc10/regex-1.0.0-rc10.jar (2062 KB at 1219.7 KB/sec)

Can you please verify you have such a file, and it is used by your setup? Graal.js depends on that maven package: https://mvnrepository.com/artifact/org.graalvm.regex/regex/usages

I have not tested the example on Windows yet.

-- Christian

[virtualzero2]

Christian:

My build is using rc12.

regex-1.0.0-rc12.jar exists in my local repo and is present in my jar. I use maven-assembly-plugin to put everything in a single JAR.

As I said in my original post everything runs fine from IntelliJ. It is only when I package everything together that I encounter the exception.

Thanks for the reply,

Dana

[pmlopes]

The issue is that the graal languages "js" and "regex" both contain a propery file with the same name and location, so when you do an uber-jar one override the other and breaks. You should either write a transformer to merge the 2 or relocate the sources.

You cannot just append the 2 because the properties are index based and both files declare the languages at index 0.

[pmlopes]

Another thing I remember from this when I've discussed it a while ago... uber-jars will give you even more trouble once you run your code on JDK11, because if the upstream jars include "module-info.class" files they will most likely shade each other breaking the exports/imports of the java module system.

I've discussed this issue with @chumer back in December and in order to embrace the future, the best is to have either your jars on the classpath/modulepath as this will even allow you to do jlink images.

For reference here is where this issue was reported to es4x: https://github.com/reactiverse/es4x/issues/57

[virtualzero2]

pmlopes:

Thanks for the feedback. I removed the jar assembly and I've added the jars to the classpath and that seems to have worked.

I had built a JRE using jlink and in order for this to work with my JRE I had to add java.management and jdk.management to it. I'll have to give this a bit more thought as I have a working knowledge at best of the proper ways to package and application.

Regards.

[aldocano29]

I am experiencing the same issue at the moment. I used graaljs inside reast api (in a war) and it worked great. Then i tried to build a jar using thesame js file and got this issue...it runs inside the IDE, but as soon as I put all the code and dependencies in a jar in breaks

[chumer]

This is less than ideal at the moment as we are currently using this custom registration mechanism for languages at the moment. We plan to switch to a different more standard service provider approach, that uber-jar will be able to handle without further configuration. It is a bigger change though.

[aldocano29]

In the end, for the time being, referencing the libraries using the classpath and not using a fat-jar/uber-jar was the solution for this issue (as mentioned above). It may not be "ideal" but it works, and (at least in my case) I needed it to work. Have a great day!

[chumer]

Ok good, thanks. It is on our radar to fix.

[nhoughto]

Also hit this issue, attempted to merge the properties file but couldn't figure out how to? Is there some doco or example of how the truffle files should be merged to make it work? Have a half written shadow plugin transformer that I can plug it into if I know what i'm trying to create =)

Or is the intent that this truffle property loading will be replaced with something that just works OTB so just wait for that? 🤞

[warrenc5]

Quick work around. Clobber shade back with your own file!

Step 1. create a file in your src/main/resources/META-INF/truffle/language
Step 2. merge the files manually by adding the entries from js-19.0.0.jar/META-INF/truffle/language and regexp-19.0.0/META-INF/truffle/language
Step 3. In your new language properties file, update the keys by renaming the key prefix from language1 to language2

[devmattrick]

@warrenc5 could you post an example of your src/main/resources/META-INF/truffle/language file? It doesn't seem to be working for me from my understanding :(

[nhoughto]

Would love an example, I spent half a day on it and couldn’t make the merged files work 🤷🏻‍♂️
On 21 Jun 2019, 5:38 PM +1000, Matthew McCune [e-mail], wrote:

@warrenc5 could you post an example of your src/main/resources/META-INF/truffle/language file? It doesn't seem to be working for me from my understanding :(
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[warrenc5]

I didn't have to update shade or jar plugin at all just added the file.

src/main/resources/META-INF/truffle/language

#https://github.com/graalvm/graaljs/issues/125

language1.characterMimeType.0=application/tregex
language1.className=com.oracle.truffle.regex.RegexLanguage
language1.id=regex
language1.implementationName=
language1.interactive=false
language1.internal=true
language1.name=REGEX
language1.version=0.1

language2.characterMimeType.0=application/javascript
language2.characterMimeType.1=application/javascript+module
language2.characterMimeType.2=text/javascript
language2.className=com.oracle.truffle.js.lang.JavaScriptLanguage
language2.defaultMimeType=application/javascript
language2.dependentLanguage.0=regex
language2.fileTypeDetector0=com.oracle.truffle.js.lang.JSFileTypeDetector
language2.id=js
language2.implementationName=GraalVM JavaScript
language2.interactive=true
language2.internal=false
language2.name=JavaScript
language2.version=inherit

hth!

[devmattrick]

@warrenc5 thanks for the response! That's what I had but it still didn't work, as it turns out I actually had some issues with the ClassLoader.

@nhoughto if you're running Graal in a plugin environment you might want to check out #182 and this StackOverflow question. It was pretty much the only way I could get it to run in my environment.

[nhoughto]

Thanks for the tip, i don't have classloader complexities, so just self merging the language file and adding it to the resources source as above worked for me.

I wonder what i was doing wrong in my shadow transformer 🤷‍♂

Either way this likely could be closed?

[nhoughto]

I'm sure there is a nice, more concise way to do this, but my ultra hacky groovy transformer for truffle language files, bit better than hard coding it, not sure what i was doing wrong before but this works:

shadowJar {
    mergeServiceFiles()
    transform(GraalTransformer.class)
}

class GraalTransformer implements Transformer {

    private final PatternSet patternSet = new PatternSet().include("META-INF/truffle/language")
    private int matchCount = 0
    private StringBuilder result = new StringBuilder()
    private String targetPath = null

    boolean canTransformResource(FileTreeElement element) { 
        return patternSet.asSpec.isSatisfiedBy(element)
    }

    void transform(TransformerContext context) {
        matchCount += 1
        targetPath = context.path
        def lines = context.is.readLines()
        lines.eachWithIndex { String line, int i ->
            def replacement = lines[i].replace("language1", "language" + matchCount)
            lines[i] = replacement
            result.append(replacement + "\n")
        }
    }

    boolean hasTransformedResource() { matchCount > 0 }

    void modifyOutputStream(ZipOutputStream os, boolean preserveFileTimestamps) {
        ZipEntry entry = new ZipEntry(targetPath)
        entry.time = TransformerContext.getEntryTimestamp(preserveFileTimestamps, entry.time)
        os.putNextEntry(entry)
        os.write(result.toString().getBytes("UTF-8"))
        os.closeEntry()
//        print("Wrote: \n${result.toString()}\n")
    }
}

[twoodhouse]

I had been running warrenc5's truffle/language file override successfully until upgrading to version 19.3.0.2

After that upgrade, I get an exception about registering duplicate language id "js"
I presume this has to do with a change in how the language registration works?

[nhartner]

I had been running warrenc5's truffle/language file override successfully until upgrading to version 19.3.0.2

After that upgrade, I get an exception about registering duplicate language id "js"
I presume this has to do with a change in how the language registration works?

Just change language2.id=js to language2.id=javascript

[twoodhouse]

I had been running warrenc5's truffle/language file override successfully until upgrading to version 19.3.0.2
After that upgrade, I get an exception about registering duplicate language id "js"
I presume this has to do with a change in how the language registration works?

Just change language2.id=js to language2.id=javascript

That would probably work, but wouldn't it make the same language registered twice under different names? That doesn't seem like the right solution.

[chumer]

With 19.3 you should no longer need to do that. Language registration now works using the standard service loader mechanism that uberjar should already be able to merge.

From the changelog:

Truffle languages and instruments no longer create META-INF/truffle files, but generate service implementations for TruffleLanguage.Provider and TruffleInstrument.Provider automatically. Recompiling the TruffleLanguage using the Truffle annotation processor automatically migrates the language.

[aldocano29]

I can confirm, with the actual version this error doesn't show up anymore. My project , which uses an Uber Jar is working without any issues at all

[vijay-garg]

Hi, guys still facing this issue for the latest version. Any help would be appreciated!

[Lewenhaupt]

Hi, guys still facing this issue for the latest version. Any help would be appreciated!

If you are using a shaded jar with maven-shade you can try adding a ServiceTransformer: https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer

This solved it for me as otherwise the services overwrote eachother, i.e. pretty much the original issue here, just a different flavor.

[Harinath2306]

Hi,

I have a similar project which builts jar using graavm. Im facing similar error "java.lang.IllegalStateException: No language for id regex found. Supported languages are: [js]" while run using jar but it works on IntelliJ.

This project works fine when run on IntelliJ but gives error when run using Jar

GraalVmDependency.zip

I don't have any file as language in my project. Can you please help me to make the necessary changes to resolve this issue. Attaching the project for reference.

Any help would be appreciated. Thanks in Advance.

[warrenc5]

looks like we have switched to the META-INF/services/com.truffle.api.TruffleLanguage$Provider (which is GREAT)

So just use the maven shade plugin transformer to merge the contents of those files

transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"

will produce.

com.oracle.truffle.js.lang.JavaScriptLanguageProvider
com.oracle.truffle.regex.RegexLanguageProvider

happy graal'n