Bloop: Unexpected error when compiling root after upgrading to 1.4.0

[danking]

I killed the bloop process listed by ps aux | grep bloop. I removed the .bloop folder. I ran sbt bloopInstall on a project that otherwise works with sbt and then I asked bloop to compile the project and got this error message.

[E] Unexpected error when compiling root: 'Unexpected exception when computing compile inputs for root'
[E] Failed to compile 'root'

How can I get more detailed information? I don't know what --debug does but I couldn't get it to work. --verbose doesn't seem to help. The project itself is rather larger and constructing a small reproducible example will be difficult. I'd prefer to learn how to tell bloop to show me what the issue is.

# rm -rf .bloop      
# sbt bloopInstall
[info] Loading settings for project global-plugins from plugins.sbt ...
[info] Loading global plugins from /Users/dking/.sbt/1.0/plugins
[info] Loading settings for project hail-build from plugins.sbt,metals.sbt ...
[info] Loading project definition from /Users/dking/projects/hail/hail/project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to hail (in build file:/Users/dking/projects/hail/hail/)
[success] Generated .bloop/root.json
[success] Generated .bloop/root-test.json
[success] Total time: 1 s, completed May 11, 2020 6:44:00 PM
# bloop compile --verbose root 
No server running at 127.0.0.1:8212, let's fire one...
Resolving ch.epfl.scala:bloop-frontend_2.12:1.4.0...
Starting bloop server at 127.0.0.1:8212...
Attempting a connection to the server...
Attempting a connection to the server...
Attempting a connection to the server...
Attempting a connection to the server...
[D] Loading 2 projects from '/Users/dking/projects/hail/hail/.bloop'...
[D] Loading project from '/Users/dking/projects/hail/hail/.bloop/root.json'
[D] Loading project from '/Users/dking/projects/hail/hail/.bloop/root-test.json'
[D] Cache miss for scala instance org.scala-lang:scala-compiler:2.11.12.
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.3/jline-2.14.3.jar
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.11.12/scala-compiler-2.11.12.jar
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar
[D]   => /Users/dking/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar
[D] Loading previous analysis for 'root-test' from '/Users/dking/projects/hail/hail/target/streams/test/bloopAnalysisOut/_global/streams/inc_compile_2.11.zip'.
[D] Loading previous analysis for 'root' from '/Users/dking/projects/hail/hail/target/streams/compile/bloopAnalysisOut/_global/streams/inc_compile_2.11.zip'.
[D] Computing sources and classpath hashes for root
[E] Unexpected error when compiling root: 'Unexpected exception when computing compile inputs for root'
[E] Failed to compile 'root'
[D] Elapsed: 1092.785509 ms
# 

[davesmith00000]

Additional information.

I'm getting the same error when I try a CLI bloop compile with 1.4.0.

[E] Unexpected error when compiling <insert project name>: 'Unexpected exception when computing compile inputs for <insert project name>'
[E] Failed to compile '<insert project name>'

Not just on root but on a whole host of projects.

Downgrading* the bloop server version from 1.4.0 to 1.4.0-RC3 "fixes" normal compilation but test compiles continue to fail as above. Downgrade to 1.4.0-RC1 and everything is working again, perfectly.

In my case, this is true for both SBT and Mill builds (tried both).

Similar to @danking, I've tried clearing cache folders, killing bloop processes, uninstalling and reinstalling via both Coursier and Homebrew (I'm on a Mac... Mojave if it matters).

As Dan says, there's not useful information I can see to work out what might be making it unhappy.

*The _only_ way I could find to downgrade was that thankfully VSCode lets to specify a version, and magically sorts it out. I can't find an official way to reinstall old versions - is there one?

Set up info:

• Metals 0.9.0
• Scala version in projects 2.13.2
• Scala.js version in projects 1.0.1
• SBT 1.3.10
• Mill 0.6.2

[Duhemm]

I ran into a similar issue. Wiping out my project's .bloop and re-creating the configuration files fixed the problem for me.

[jvican]

I recommended in gitter to run bloop exit as well, it hasn't worked for some folks but for others it has. I don't know why this issue has happened now. We've been testing all changes exhaustively locally. I don't think there's a code change made between 1.4.0 and 1.4.0-RC3, so I'm wondering if this is just a spurious issue.

[davesmith00000]

Just for completeness:
@Duhemm I have tried that but the problem remains.
@jvican 1.4.0-RC3 didn't seem to work for me either, only 1.4.0-RC1 works as expected - I don't know if that helps. Was there an RC2 I could test for you?

[jvican]

Sorry, can you confirm you have run bloop exit, you have removed $CWD/.bloop and then you run bloop about and the issue remains? bloop about should print "bloop 1.4.0". I just want to be 100% sure.

There are only two changes I have not tested locally in my machine before cutting 1.4.0: upgrading to the latest Coursier RC and https://github.com/scalacenter/bloop/pull/1262. I wonder if this last PR has something to do with this misbehavior.

[davesmith00000]

No worries! :-)

So just for you, I have now:

• Put my machine back into a broken state with 1.4.0, checked both editor wasn't responding and that I couldn't bloop compile <project>.
• bloop exit
• rm -fr .bloop/
• rm -fr .metals/
• bloop about (starts up bloop again "let's fire one...")
• bloop compile <project> fails due to lack of project files
• sbt bloopInstall to generate
• bloop compile <project> fails in the described way, see below.

> bloop compile <project>
[E] Unexpected error when compiling <project>: 'Unexpected exception when computing compile inputs for <project>'
[E] Failed to compile '<project>'

• Switch back to good-old 1.4.0-RC1 and everything immediately comes back to life.

Might be worth pointing out that this is an SBT multi project crossproject too. So it's building multiple JVM and JS versions of libraries that form respective dependency chains. There are 40 bloop json files generated including test versions.

[jvican]

@davesmith00000 Thanks for that. I still can't reproduce it on my own. Can you try with 1.4.0-RC1-229-b7c15aa9? Also, what bloop version is your sbt project using? Have you upgraded sbt-bloop? What happens if you do?

[jvican]

Alright, seems like the coursier upgrade is causing this error. I'm reverting that upgrade in https://github.com/scalacenter/bloop/pull/1277 and will cut a release tonight.

In the meanwhile, for those wanting to try 1.4.0 out, I recommend following these instructions:

• Checkout https://github.com/scalacenter/bloop/pull/1277
• Run sbt install
• Install bloop via the brew, arch or the scoop formulas generated by sbt install, or directly via coursier (bloop should be already in your ivy2 local cache)

You might need to have graalvm installed in your environment and not use Windows (it's too painful to generate a graalvm native image).

The contributing guide will get everyone of you familiar with the project in case you need it https://scalacenter.github.io/bloop/docs/contributing-guide

[jvican]

If nobody wants to build bloop locally, you can use 1.4.0-RC1-237-80d969bb which it's also a good version very close to the end product of 1.4.0. Remember to upgrade sbt-bloop as well.

[davesmith00000]

@jvican Thank you very much for looking into this. I can confirm that both those versions work:

You asked me to test this version, and it does work.
1.4.0-RC1-229-b7c15aa9

Your recommendation also does appears to work just fine.
1.4.0-RC1-237-80d969bb

I'm using the same sbt-bloop version as the bloop server in all cases.

[danking]

Wow, this was a much faster response than I anticipated! Thank you all, particularly @jvican!

I cloned sbt, checked out that branch, ran git submodule init, and ran sbt install. I think I ran into this issue: https://github.com/sbt/sbt/issues/4857. I copied the output to the end of this post. I tried adding sbtPlugin := true to both sbtBloop10 and sbtBloop10Shaded. The version of SBT I have installed (from brew) is 1.3.10.

I also apparently don't understand courier, because I ran this:

coursier install bloop:1.4.0-RC1-237-80d969bb

Which created a bloop binary that appears to not work

# '/Users/dking/Library/Application Support/Coursier/bin/bloop'
Error: Could not find or load main class bloop.bloopgun.Bloopgun

I don't know if that's a JAR but it does appear quite empty

# jar tf '/Users/dking/Library/Application Support/Coursier/bin/bloop' 

META-INF/coursier/info.json
META-INF/coursier/lock-file
META-INF/coursier/info-source.json

I have coursier 2.0.0-RC6-15. When I use coursier to install bloop:1.4.0 it indeed gives me a working bloop 1.4.0 binary that reproduces the original error.

I get the same "Could not find or load main class" issue with the bloop binary when I install 1.4.0-RC1: coursier install bloop:1.4.0-RC1.

Thanks for your ongoing support. I'm happy to test anything else that may help verify the fix.

# sbt install
[info] Loading settings for project global-plugins from plugins.sbt ...
[info] Loading global plugins from /Users/dking/.sbt/1.0/plugins
[info] Loading settings for project bloop-build-build-build from build.sbt ...
[info] Loading project definition from /Users/dking/projects/bloop/project/project/project
[info] Loading settings for project root from build.sbt ...
[info] Loading project definition from /Users/dking/projects/bloop/project/project
[info] Wrote /Users/dking/projects/bloop/project/project/sbt-bloop-build-shaded-naked/target/scala-2.12/sbt-bloop-build-shaded-naked_2.12-1.0.0-SNAPSHOT.pom
[warn] Multiple main classes detected.  Run 'show discoveredMainClasses' to see the list
[info] Updating 
[info] Resolved  dependencies
[warn] 
[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]      com.dwijnand:sbt-compat:1.0.0 (scalaVersion=2.12, sbtVersion=1.0)
[warn] 
[warn]  Note: Unresolved dependencies path:
[info] Found 6964 class(es) in JAR(s) to be shaded
[info] 0 remaining class(es) to be shaded
[error] sbt.librarymanagement.ResolveException: Error downloading com.dwijnand:sbt-compat;sbtVersion=1.0;scalaVersion=2.12:1.0.0
[error]   Not found
[error]   Not found
[error]   not found: /Users/dking/.ivy2/local/com.dwijnand/sbt-compat/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/dwijnand/sbt-compat_2.12_1.0/1.0.0/sbt-compat-1.0.0.pom
[error]     at lmcoursier.CoursierDependencyResolution.unresolvedWarningOrThrow(CoursierDependencyResolution.scala:245)
[error]     at lmcoursier.CoursierDependencyResolution.$anonfun$update$34(CoursierDependencyResolution.scala:214)
[error]     at scala.util.Either$LeftProjection.map(Either.scala:573)
[error]     at lmcoursier.CoursierDependencyResolution.update(CoursierDependencyResolution.scala:214)
[error]     at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:60)
[error]     at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:52)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:102)
[error]     at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:69)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$20(LibraryManagement.scala:115)
[error]     at scala.util.control.Exception$Catch.apply(Exception.scala:228)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:115)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:96)
[error]     at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:150)
[error]     at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:129)
[error]     at sbt.Classpaths$.$anonfun$updateTask0$5(Defaults.scala:2946)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error]     at sbt.std.Transform$$anon$4.work(Transform.scala:67)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:281)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:19)
[error]     at sbt.Execute.work(Execute.scala:290)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:281)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]     at java.lang.Thread.run(Thread.java:748)
[error] (sbtBloopBuildShadedNakedJar / update) sbt.librarymanagement.ResolveException: Error downloading com.dwijnand:sbt-compat;sbtVersion=1.0;scalaVersion=2.12:1.0.0
[error]   Not found
[error]   Not found
[error]   not found: /Users/dking/.ivy2/local/com.dwijnand/sbt-compat/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/dwijnand/sbt-compat_2.12_1.0/1.0.0/sbt-compat-1.0.0.pom
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 

[jvican]

Thanks for that information @danking and thank you everyone who chimed in. I've just released v1.4.1 which should have this issue fixed. The root cause behind this issue is that 1.4.0 is not working for Java 8 users, but it is working for Java 11 users. This error was caused by a GraalVM version upgrade which changed its default from Java 8 to Java 11. As a result, our build process released our artifacts with Java 11. Please take into account v1.4.1 might not yet be available as it needs to hit Maven Central first.