Metals: Speed up metalsSetup command

[fommil]

I just tried to run the metals sbt task on my work project and it was taking so long I had to cancel it.

I know exactly what the problem is, because I just looked into the MetalsPlugin, and we hit this in ensime about 3 years ago. You need to do a single updateClassifiers and then cache the results and lookup from there. This is the approach taken in ensime, intellij and scalaide plugins.

If we are building a shared "build structure" plugin it would avoid duplication of this logic, which is fiddling and easy to get wrong. Your scala version detection is also a bit ropey because you're assuming the user knows how to create a clean build definition. We have an election and pick the winner.

BTW you could have saved yourself a lot of time by simply depending on sbt-ensime and calling our task to create an EnsimeConfig object (just a case class) then serialising it whatever way you wanted. Adding a JSON writer would only take a few minutes.

[fommil]

btw, here are a couple of projects that are suitable for testing your algorithm on, if you choose to continue with your own solution

• https://gitlab.com/fommil/attic/tree/master/sbt-big-project/src/sbt-test/sbt-big-project/clusterfsck
• https://github.com/cakesolutions/sbt-cake/tree/sbt-perf-regression

[laughedelic]

Thanks for the suggestions! Could you give me some numbers? I tried it on the sbt-cake project and saw this:

• updateClassifiers takes >15min, downloading half of the internet. I thought to stop it at some point but then just went for a coffee.
• metalsSetup takes ~35-45s and I think that semanticdbEnable could be improved to speed it up
• */metalsWriteBuildInfo takes ~15-20s, which I think is reasonable for a build with 26 subprojects

You need to do a single updateClassifiers and then cache the results and lookup from there. This is the approach taken in ensime, intellij and scalaide plugins.

I didn't have time yet to look into ensime-sbt, but I'm a bit confused about this: isn't updateClassifiers cached by sbt between the runs of metalsWriteBuildInfo for different projects?

If we are building a shared "build structure" plugin it would avoid duplication of this logic, which is fiddling and easy to get wrong.

IMO it's a bit early for this shared thing. We're not trying to solve all possible problems here, just providing minimum required information for Metals. So if later such a shared sbt plugin and config format appears, we will gladly migrate to it. At the moment there are several such formats and I don't think that Ensime's is the optimal one for all consumers of it.

Your scala version detection is also a bit ropey because you're assuming the user knows how to create a clean build definition.

Doesn't scalaVersion have some default set in sbt? Or what is the problem you're referring to?

gitlab.com/fommil/attic/tree/master/sbt-big-project/src/sbt-test/sbt-big-project/clusterfsck

This refers to a scripted test (which doesn't load as a standalone sbt project). Did you mean sbt-big-project?

[fommil]

If we go down the route of porting the ensime workarounds, we're just reinventing the wheel without any new approach to the problem. For now, the easiest thing for you would be to call sbt-ensime and convert the project definition.

Strategically, all authors (scalaide, intellij, ensime, metals, sbt) could combine and create a common library (and potentially file format) for doing this that includes all the workarounds we've each come up with.

There are already some intellij plugins that try to get the conversation started.