Bloop: Unexpected `CreateProcessW` error in Windows

[schleumer]

There's an issue i've found running my project on Windows, when running(bloop run) large projects bloop generates really big command which causes CreateProcessW to reject the execution thus making bloop run fails, command limitation is 32768 characters, bloop generated one with 44540 characters for my project. It's actually just a report of a compatibility issue on Windows, running it on Linux just runs fine.

[jvican]

Thanks for reporting! @schleumer Is this something you can look into and provide a fix? I can give you pointers, I think I know where this is happening.

[schleumer]

I forgot do add the stack trace:

java.lang.RuntimeException: CreateProcessW() failed, error: 206
        at com.zaxxer.nuprocess.windows.WindowsProcess.start(WindowsProcess.java:248)
        at com.zaxxer.nuprocess.windows.WinProcessFactory.createProcess(WinProcessFactory.java:42)
        at com.zaxxer.nuprocess.NuProcessBuilder.start(NuProcessBuilder.java:266)
        at bloop.exec.Forker$.$anonfun$run$6(Forker.scala:183)
        at monix.eval.internal.TaskRunLoop$.monix$eval$internal$TaskRunLoop$$loop$1(TaskRunLoop.scala:187)
        at monix.eval.internal.TaskRunLoop$RestartCallback$1.onSuccess(TaskRunLoop.scala:119)
        at monix.eval.Task$.$anonfun$forkedUnit$2(Task.scala:1463)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1425)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

Also:

OpenJDK 12
Scala(global) 2.13.1
Scala(local) 2.12.9
SBT(global) 1.3.3
SBT(local) 1.3.0

And the command generated by Bloop:

https://gist.github.com/schleumer/6006c154dc1708872457afeb1638b586

If you can point me, i can work it out as soon as possible.

[jvican]

Thanks for providing the additional information, will be important for people running into the same issue.

The line where we generate the java command to run/test a project is this one:
https://github.com/scalacenter/bloop/blob/f5e1cfe2ccb63b3cab12d20a6e4e611b981b01c4/frontend/src/main/scala/bloop/exec/JvmProcessForker.scala#L119

I've run into this error before and I don't remember what I did to fix this, but I do remember that there were several StackOverflow answers that provided workarounds for this issue. I believe we should find the work around that makes the most sense in Windows and implement it in the linked file.

[schleumer]

I found a way that involves the creation of a temporary file containing all the args and passing to Java executable, this will be acceptable only on Java 9+. Also, from the workarounds I found, none worked, even the one to put all files on root directory to reduce class path characters. I've edited the previous comment to add the command generated by the bloop run.

[jvican]

Thanks for looking into this!

The first workaround seems good to me. I propose we do that only in Windows machines and only once the character limit has been surpassed. If the workaround fails because, for example, the user doesn't use Java 9 (which is likely), can we detect the exception and throw a helpful message pinpointing what the issue is and recommending them to use Java 9 to run that project?

this will be acceptable only on Java 9+

Some official docs pointing this out would be helpful, maybe we can even add them in the message shown to the user.

[jvican]

@schleumer I've just gotten a report by @olafurpg in https://github.com/scalacenter/bloop/issues/1094 where this issue seems to be cross-platform. We've bounced off some ideas about how to solve this. It looks like the best solution is to create a simple jar file with only one MANIFEST file in it that contains the Class-Path: attribute and perhaps also the main class.

I think this solution is superior to the work-around discussed in the ticket and should be straightforward to implement by using Java NIO FileSystem APIs.

[schleumer]

@jvican, sorry for the delay, you have some material that i can follow about the JAR creation? This is kinda new for me, but with some resource would be easy.

[jvican]

@schleumer I think @olafurpg is already looking into this, so let's close this ticket in favor of #1094.