Jabref: Entry preview: Error while generating citation style

[KarlMagnusLarsson]

JabRef 5.0.0-dev--2019-11-16----3fbeef03d
Mac OS X 10.13.6 x86_64
Java 13.0.1

Steps to reproduce:

1. Select any entry from an open bibliography.
2. Double left click on entry to show the different fields in the lower portion of the window.
3. The right hand side shows "Error while generating citation style", instead of a preview of the citation.

Jabref-5.0.0 has been in this state for a couple of development snapshots.

I have configured APA style, but I get the same "Error while generating citation style" with other styles. I get the same error for all entries in the bibliography.

[Siedlerchr]

Can confirm this. Think it is a follow up issue from #5585
I debugged it down and it triggers the ArgumentException which is ignored.

Actual exception and stack trace.
I looked into my JabRef folder and it seems like there is some problem with the csl-locale git submodule. Mine is empty.

org.graalvm.polyglot.PolyglotException: Unable to load locale en-US
 [org.jabref/org.jabref.logic.citationstyle.JabRefLocaleProvider.lambda$0(JabRefLocaleProvider.java:29), java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1138), org.jabref/org.jabref.logic.citationstyle.JabRefLocaleProvider.retrieveLocale(JabRefLocaleProvider.java:25), <js>.Sys.retrieveLocale(Unnamed:19), <js>.localeConfigure(Unnamed:9044), <js>.CSL.Engine(Unnamed:3710), <js>.makeCsl(Unnamed:54), org.graalvm.sdk/org.graalvm.polyglot.Value.execute(Value.java:366), [email protected]/de.undercouch.citeproc.script.GraalScriptRunner.callMethod(GraalScriptRunner.java:49), [email protected]/de.undercouch.citeproc.CSL.<init>(CSL.java:283), [email protected]/de.undercouch.citeproc.CSL.<init>(CSL.java:252), [email protected]/de.undercouch.citeproc.CSL.<init>(CSL.java:231), org.jabref/org.jabref.logic.citationstyle.CSLAdapter.initialize(CSLAdapter.java:69), org.jabref/org.jabref.logic.citationstyle.CSLAdapter.makeBibliography(CSLAdapter.java:53), org.jabref/org.jabref.logic.citationstyle.CitationStyleGenerator.generateCitations(CitationStyleGenerator.java:56), org.jabref/org.jabref.logic.citationstyle.CitationStyleGenerator.generateCitation(CitationStyleGenerator.java:47), org.jabref/org.jabref.logic.citationstyle.CitationStylePreviewLayout.generatePreview(CitationStylePreviewLayout.java:15), org.jabref/org.jabref.gui.preview.PreviewViewer.lambda$4(PreviewViewer.java:184), org.jabref/org.jabref.gui.util.BackgroundTask$1.call(BackgroundTask.java:47), org.jabref/org.jabref.gui.util.DefaultTaskExecutor$1.call(DefaultTaskExecutor.java:130), javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Task.java:1425), java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264), java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515), java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264), java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128), java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628), java.base/java.lang.Thread.run(Thread.java:830)]

[Siedlerchr]

After I deleted and readded the csl-locale submodule it works again for me when running from eclipse.
So I guess there might be something wrong with the modules in the installer

[tobiasdiez]

@koppor what do you think about replacing the git submodules by git subtrees? Then we also don't run into these problems that some people are pushing changes to the submodule commit hash as part of PRs.

[koppor]

Reading https://codewinsarguments.co/2016/05/01/git-submodules-vs-git-subtrees/ -- +1 for subtrees.

Is the external repository third party code that you are unlikely to push anything back to? Then use a subtree. This gives the advantage of not having to give people permissions to an extra repo when you are giving them access to the code base, and also reduces the chance that someone will forget to run a git submodule update.

Should we go for the "squashed" history? --> https://stackoverflow.com/a/51774939/873282 --> "Alt Squashed history"

[tobiasdiez]

Yes, squashed history sounds good (I don't think we care about the detailed changes to the style files).

[koppor]

PR to switch to git subtrees created: https://github.com/JabRef/jabref/pull/5647

[Siedlerchr]

Git subtrees fixed the thing with the missing resources. Now I get the following error in the portable version of JabRef with loading the csl:

ERROR PreviewViewer Error while generating citation style
 java.lang.Exception: java.lang.NoClassDefFoundError: org/graalvm/polyglot/PolyglotException
        at org.jabref/org.jabref.gui.util.DefaultTaskExecutor.convertToException(Unknown Source)
        at org.jabref/org.jabref.gui.util.DefaultTaskExecutor.lambda$getJavaFXTask$3(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEventImpl(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEvent(Unknown Source)
        at org.jabref.merged.module/javafx.event.Event.fireEvent(Unknown Source)
        at org.jabref.merged.module/javafx.concurrent.EventHelper.fireEvent(Unknown Source)
        at org.jabref.merged.module/javafx.concurrent.Task.fireEvent(Unknown Source)
        at org.jabref.merged.module/javafx.concurrent.Task.setState(Unknown Source)
        at org.jabref.merged.module/javafx.concurrent.Task$TaskCallable.lambda$call$2(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at org.jabref.merged.module/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(Unknown Source)
        at org.jabref.merged.module/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
        at org.jabref.merged.module/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
        at org.jabref.merged.module/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: org/graalvm/polyglot/PolyglotException
        at org.jabref.merged.module/de.undercouch.citeproc.script.ScriptRunnerFactory.createGraalRunner(Unknown Source)
        at org.jabref.merged.module/de.undercouch.citeproc.script.ScriptRunnerFactory.createRunner(Unknown Source)
        at org.jabref.merged.module/de.undercouch.citeproc.CSL.getRunner(Unknown Source)
        at org.jabref.merged.module/de.undercouch.citeproc.CSL.<init>(Unknown Source)
        at org.jabref.merged.module/de.undercouch.citeproc.CSL.<init>(Unknown Source)
        at org.jabref.merged.module/de.undercouch.citeproc.CSL.<init>(Unknown Source)
        at org.jabref/org.jabref.logic.citationstyle.CSLAdapter.initialize(Unknown Source)
        at org.jabref/org.jabref.logic.citationstyle.CSLAdapter.makeBibliography(Unknown Source)
        at org.jabref/org.jabref.logic.citationstyle.CitationStyleGenerator.generateCitations(Unknown Source)
        at org.jabref/org.jabref.logic.citationstyle.CitationStyleGenerator.generateCitation(Unknown Source)
        at org.jabref/org.jabref.logic.citationstyle.CitationStylePreviewLayout.generatePreview(Unknown Source)
        at org.jabref/org.jabref.gui.preview.PreviewViewer.lambda$update$3(Unknown Source)
        at org.jabref/org.jabref.gui.util.BackgroundTask$1.call(Unknown Source)
        at org.jabref/org.jabref.gui.util.DefaultTaskExecutor$1.call(Unknown Source)
        at org.jabref.merged.module/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        ... 1 more
Caused by: java.lang.ClassNotFoundException: org.graalvm.polyglot.PolyglotException
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
        at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
        ... 21 more

[tobiasdiez]

Everything works if you run from code (Intellj). However, when using the installer I still have an empty list of citation styles in the preference dialog.

[Siedlerchr]

When you run from installer or portable the exception above is logged on the console window behind JabRef.
From my point of view this looks like a modularity issue. Maybe some module access stuff

[tobiasdiez]

@michel-kraemer can you please have a look at this, as this seems to be a problem with citeproc 2.0. Thanks!

[koppor]

This could be a solution? https://github.com/JabRef/jabref/issues/5246#issuecomment-529208270

[michel-kraemer]

It seems to me org.graalvm.polyglot.PolyglotException is missing in the portable version. It is located inside graal-sdk-19.2.1.jar. Can you please check if this file is there? Or, if you're creating a fat jar, if the class file org/graalvm/polyglot/PolyglotException.class is there?

@koppor I don't see how this could be a solution. By the way, there is a method called CSL.getSupportedStyles() that returns a list of all styles in the classpath. It should work regardless of whether the application is run from the IDE or from a packaged jar file.

[michel-kraemer]

Oh, by the way, I'm currently working on a pure Java implementation of citeproc-java. It does not support all CSL features yet but it works OK most of the time already. If you're interested, I can publish a snapshot of citeproc-java. This should solve all problems related to Graal JavaScript or Nashorn, and it should improve performance by a magnitude.

[Siedlerchr]

@michel-kraemer
The PolyglotException class is there: I checked again with the latest master (windows portable)
https://builds.jabref.org/master/
JabRef-5.0-portable_windows\JabRef\app\modsgraal-sdk-19.2.1.jar\org\graalvm\polyglot\

The only thing which comes to my minds is, do you have an (Automatic) Module name/manifest for citeproc-java? Maybe it's some modularity/classpath issue with jlink/jpackage.

I think it's worth a try to test the snapshot of the pure java implementation

[Siedlerchr]

@michel-kraemer Any news on this? We are planning a new release and it would be great to check if it works with a snapshot of your java implementation

[KarlMagnusLarsson]

Works fine from current (19-feb-2020) code, if you build from code, as was pointed out by @tobiasdiez 6-dec-2019.

1: clone repo
2: cd jabref
3: gradlew run
4: Open bib-file and pick preview APA7 or IEEE => preview is fine.

Source:
JabRef 100.0.0
Mac OS X 10.13.6 x86_64
Java 13.0.1
commit e560220bc74430cbeeb58838f9b0db75a6555a03

Current development binary 5.0 does not work for me, with same error as before for APA and IEEE (internal view "preview" is fine).

Development binary:
JabRef 5.0-beta.432--2020-02-19--c768697
Mac OS X 10.13.6 x86_64
Java 13.0.2

[michel-kraemer]

@michel-kraemer Any news on this? We are planning a new release and it would be great to check if it works with a snapshot of your java implementation

Sorry for the delay. Sure, I can publish a new snapshot with the latest changes and make a pull request on the weekend, if that's fine with you. I'll keep you updated.

[michel-kraemer]

I just created the pull request. The new pure-Java mode is super fast and seems to work well.

Please note that the snapshot does not support all features of CSL yet (CSL is extremely complex to implement) but at least the most important ones (about 80-90% of the specification). I'm continuously working on improving the library and will release a final version 2.1.0 in the next couple of months.

Also note that the snapshot supports text and HTML output only at the moment. I found a comment in the code of CSLAdapter saying that the output format could be set to RTF. I don't know if that's true or if text and HTML are fine for you.

By the way, I also tried to fix the installer by adding the required modules to the jlink configuration in the build.gradle but to no avail. 🤷‍♂

[tobiasdiez]

Thanks @michel-kraemer, that are really good news!

@koppor @Siedlerchr I tried this version (in the portable version) and the "Entry preview" panel in the preference is still empty for me, no citation styles are loaded. No error messages are logged.

[koppor]

I did gradlew run and had a list of all citations in the JabRef preferences
to choose from.

Could you push the branch to our repo to trigger a binary build
nevertheless?

Tobias Diez [e-mail] schrieb am Sa., 22. Feb. 2020, 20:09:

Thanks @michel-kraemer https://github.com/michel-kraemer, that are
really good news!

@koppor https://github.com/koppor @Siedlerchr
https://github.com/Siedlerchr I tried this version (in the portable
version) and the "Entry preview" panel in the preference is still empty for
me, no citation styles are loaded. No error messages are logged.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/JabRef/jabref/issues/5622?email_source=notifications&email_token=AAKNU7XLWTJZVT4MZQ4FUOLREFZ43A5CNFSM4JOGEUCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVIEUQ#issuecomment-589988434,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKNU7Q5KMTSUAYHMXUCUADREFZ43ANCNFSM4JOGEUCA
.

[michel-kraemer]

@koppor @Siedlerchr I tried this version (in the portable version) and the "Entry preview" panel in the preference is still empty for me, no citation styles are loaded. No error messages are logged.

That's strange. I tested it with ./gradlew run and with jpackage. No problems here.

[tobiasdiez]

I guess it's again a problem with the bundled/installed app. I've downloaded the windows builds from https://github.com/JabRef/jabref/actions/runs/43527976. The same problem occurs in the quite old build JabRef 5.0-beta.1--2019-12-15--0b8b1d6, so this appears not to be special for your PR.

[Siedlerchr]

Question is, is it jlink or jpackage which has the bug?
We can try to create a debug build: https://devdocs.jabref.org/debugging-jpackage-installations
We just need to remove the --strip-debug and can try to debug the loading and maybe get an exception

[tobiasdiez]

I'm pretty sure it's not a bug in jlink or jpackage but in our code. There is a difference in how resources are found in a local build (= they are just normal files and our FileSystem approach works) vs in a bundled build (=they are files in a jrt module and our FileSystem approach does not work).
https://github.com/JabRef/jabref/blob/1fd53bfdeb23afddbb23f4d1c18bdc6b085c2ebf/src/main/java/org/jabref/logic/citationstyle/CitationStyle.java#L146-L159

A possible solution is outlined here: https://github.com/JabRef/jabref/issues/5246#issuecomment-529208270

[Siedlerchr]

@tobiasdiez I think I maybe found a solution. https://stackoverflow.com/a/54142975
Will test it.

[tobiasdiez]

Hopefully, this should be fixed in the latest development version. Could you please check the build from http://builds.jabref.org/master/. Thanks! Please remember to make a backup of your library before trying-out this version.

[Siedlerchr]

Still present. Starting from the bat I now get a different error:

rg.graalvm.polyglot.PolyglotException: SyntaxError: Error initializing language 'regex' using class 'com.oracle.truffle.regex.RegexLanguage'.
Error initializing language 'regex' using class 'com.oracle.truffle.regex.RegexLanguage'.
Unnamed:84:14 Expected ; but found :
        "art.": "article",
              ^
Unnamed:85:13 Expected ; but found :
        "bk.": "book",
             ^
Unnamed:86:13 Expected ; but found :
        "ch.": "chapter",
             ^
Unnamed:87:16 Expected ; but found :
        "subch.": "subchapter",
                ^
Unnamed:88:12 Expected ; but found :
        "p.": "page",
            ^
Unnamed:89:13 Expected ; but found :
        "pp.": "page",
             ^
Unnamed:90:15 Expected ; but found :
        "para.": "paragraph",
               ^
Unnamed:91:18 Expected ; but found :
        "subpara.": "subparagraph",
                  ^
Unnamed:92:13 Expected ; but found :
        "pt.": "part",
             ^
Unnamed:93:12 Expected ; but found :
        "r.": "rule",
            ^
Unnamed:94:14 Expected ; but found :
        "sec.": "section",
              ^
Unnamed:95:17 Expected ; but found :
        "subsec.": "subsection",
                 ^
Unnamed:96:15 Expected ; but found :
        "supp.": "supplement",
               ^
Unnamed:97:14 Expected ; but found :
        "sch.": "schedule",
              ^
Unnamed:98:14 Expected ; but found :
        "tit.": "title",
              ^
Unnamed:99:14 Expected ; but found :
        "col.": "column",
              ^
Unnamed:100:14 Expected ; but found :
        "fig.": "figure",
              ^
Unnamed:101:14 Expected ; but found :
        "fol.": "folio",
              ^
Unnamed:102:12 Expected ; but found :
        "l.": "line",
            ^
Unnamed:103:12 Expected ; but found :
        "n.": "note",
            ^
Unnamed:104:13 Expected ; but found :
        "no.": "issue",
             ^
Unnamed:105:13 Expected ; but found :
        "op.": "opus",
             ^
Unnamed:106:13 Expected ; but found :
        "sv.": "sub-verbo",
             ^
Unnamed:107:14 Expected ; but found :
        "vrs.": "verse",
              ^
Unnamed:108:14 Expected ; but found :
        "vol.": "volume"
              ^
Unnamed:109:4 Expected eof but found }
    },
    ^

        at org.graalvm.polyglot.Context.eval(Unknown Source) ~[org.graalvm.sdk:?]
        at de.undercouch.citeproc.script.GraalScriptRunner.eval(Unknown Source) ~[org.jabref.merged.module:?]
        at de.undercouch.citeproc.script.AbstractScriptRunner.loadScript(Unknown Source) ~[org.jabref.merged.module:?]
        at de.undercouch.citeproc.CSL.getRunner(Unknown Source) ~[org.jabref.merged.module:?]
        at de.undercouch.citeproc.CSL.<init>(Unknown Source) ~[org.jabref.merged.module:?]
        at de.undercouch.citeproc.CSL.<init>(Unknown Source) ~[org.jabref.merged.module:?]
        at de.undercouch.citeproc.CSL.<init>(Unknown Source) ~[org.jabref.merged.module:?]
        at org.jabref.logic.citationstyle.CSLAdapter.initialize(Unknown Source) ~[org.jabref:?]
        at org.jabref.logic.citationstyle.CSLAdapter.makeBibliography(Unknown Source) ~[org.jabref:?]

[tobiasdiez]

This new error does not seem to be JabRef related. I guess the error message points to this part
https://github.com/Juris-M/citeproc-js/blob/3389e9b23a1873c1bb0af16eef7533aefd669701/src/load.js#L59-L85
Not sure where the problem lies. Maybe @michel-kraemer has an idea?

@michel-kraemer you might want to consider adding the relevant statements from https://github.com/JabRef/jabref/commit/d2de477d8de49de191c5fad16a65c65a96d102a3 to the usage notes of your nice citeproc-java library to make it easier for other people to use it in a modularized setting.

[michel-kraemer]

@tobiasdiez The error message is misleading. I received the same error when I tried to integrate citeproc-java with GraalVM the first time into JabRef. I solved it by adding the required modules to the module-info.java file. So, if the same error now appears in the packaged version, it's most likely still a packaging issue.

@michel-kraemer you might want to consider adding the relevant statements from d2de477 to the usage notes of your nice citeproc-java library to make it easier for other people to use it in a modularized setting.

Good idea, but I hope that I can completely get rid of GraalVM in the future when the new pure-Java implementation is ready.

[michel-kraemer]

What I mean with "the error message is misleading": It is trying to load the regex extension of Graal JavaScript but cannot find the implementation classes.

[Siedlerchr]

This issue should be fixed now in master.
Please wait until the new build is ready. That takes approx half an hour or so.

We would like to ask you to use a development build from https://builds.jabref.org/master and report back if it works for you. Please remember to make a backup of your library before trying-out this version.