Bloop: Implement the gradle integration

[jvican]

This is work in progress. It's an umbrella ticket so that interested people can subscribe to this and receive updates when the integrations is merged. /cc @time2die

[eyalroth]

I would love to know what is there to be done in order to complete this, and perhaps help out with this.

I've taken a look at the Maven integration and it seems that all that the plugin is doing is writing a configuration file for Bloop, is that correct? I guess I'm missing something because then how is Maven aware of the compilation result?

Also, is there a plan to integrate Bloop with IntelliJ?

[jvican]

I've taken a look at the Maven integration and it seems that all that the plugin is doing is writing a configuration file for Bloop, is that correct? I guess I'm missing something because then how is Maven aware of the compilation result?

Yes, writing a bloop configuration file https://scalacenter.github.io/bloop/docs/configuration-format/ is all it's required.

In the sbt integration, bloop does not share the classes directories with the stock build tool by default to avoid misinteractions. This can be changed by redefining an sbt setting, though. In Maven's case, it currently shares the classes directories. We can change that as we see fit.

I was adding some fixes to the maven integration two days ago, they are available in this branch https://github.com/scalacenter/bloop/tree/topic/add-docs-maven-plugin. I still need to work a little bit more on it, but I was trying it out in Spark's mvn build and was working :smile:.

@eyalroth If you wanna help, starting to put together a Gradle integration would be invaluable -- Maven integration was extremely tricky to get right because I used some tricks to recycle all the settings from scala-maven-plugin, but in Gradle's case all that complexity goes away because it has official Scala support. If you need live chatting, contact me at https://gitter.im/scalacenter/bloop, happy to answer any questions.

Also, is there a plan to integrate Bloop with IntelliJ?

Yes, we're working with the build tool team at IntelliJ to use the Build Server Protocol http://github.com/scalacenter/bsp/ to communicate with bloop to import projects and compile in Bloop's build server. We're moving fast in this area, so we'll have exciting progress to show soon.

[eyalroth]

I'm a little confused about this.

From the original PR of the maven integration (#148) I understand that the integration is in fact an additional maven plugin which extends from davidB's Scala Maven Plugin, and according to issue #208 on said repository I gather that the maven plugin integrates with the old (and deprecated) Zinc.

Now, as said earlier, the bloop maven integration only writes configuration files to bloop (this is done via the installBloop maven task, correct?). I'm assuming it is also possible to integrate this task / goal in the pom.xml so that Maven would update bloop with project changes (though this does not appear to be explained in Bloop's official guide).

I'm also assuming that bloop picks up on said configuration files automatically and starts compiling, but then again - how exactly maven is aware of the compilation result / status? The existing scala maven plugin doesn't seem to be communicating with the bloop server, but only with the old Zinc server.

Again, what am I missing here?

[Duhemm]

Now, as said earlier, the bloop maven integration only writes configuration files to bloop (this is done via the installBloop maven task, correct?)

Yes

I'm assuming it is also possible to integrate this task / goal in the pom.xml so that Maven would update bloop with project changes (though this does not appear to be explained in Bloop's official guide).

Do you mean automatically re-generating the Bloop configuration files when the project definition changes? I don't know Maven well enough to tell whether this is possible, but it could very well be.

how exactly maven is aware of the compilation result / status?

It is not. The idea is that you use Bloop for your daily development, and Maven only for publishing and other unfrequent tasks, so not sharing the state of the incremental compiler is not such a big deal.

I'm also assuming that bloop picks up on said configuration files automatically and starts compiling

Bloop automatically picks up changes to the configuration files, but it doesn't automatically start compiling. You still need to invoke bloop compile foobar to make it compile your project.

The existing scala maven plugin doesn't seem to be communicating with the bloop server, but only with the old Zinc server.

Indeed, there's no communication between Maven and Bloop at the moment (apart from the configuration files). When you do mvn compile, Bloop isn't involved at all in the process.

I think that your confusion comes from what I think of as "second stage integration", where running mvn compile invokes bloop compile under the hood. We don't have such an integration at the moment for any of the build tools that we support, but we plan to work on that.

For every build tool, the first step is to be able to generate the Bloop configuration file from them. This is a required step to be able to tackle the "second stage".

I hope I answered your questions, let me know if you have more!

[eyalroth]

@Duhemm Ooohhh. That explains it. I had a different idea of an integration means - I was indeed imagining what you coined as a "second stage integration". I believe that might be the first thing that comes into some people minds when reading about the integration on the official site; perhaps it is worth clarifying what this integration entails.

The idea is that you use Bloop for your daily development, and Maven only for publishing and other infrequent tasks, so not sharing the state of the incremental compiler is not such a big deal.

I would guess that for most developers the day-to-day compilation is done via their chosen IDE. In this case, I believe there's a need for a clear picture of how the three components communicate with each other, and by "three components" I'm referring to the build tool / framework, the IDE and the compiler.

I would be happy to use the same compiler on my local machine and on my CI / CD servers (perhaps just with different configurations), and since the build tool is the basis of the CI / CD, it seems to me that the build tool has to be able to communicate with the compiler. Now that leaves the question of how the IDE communicates / integrates with them both.

Also, the build tool definitely isn't used solely in the CI / CD process. I often build my applications locally. And in fact, I trust Gradle to run my tests better than I trust IntelliJ (my IDE of choice, for now); for instance, I can't configure IntelliJ to respect the file encoding of my tests which is configured in my Gradle files. Therefore, I configured IntelliJ to run tests via Gradle and not with its own testing platform.

Perhaps IntelliJ's own testing platform works better when reading build configuration of other frameworks (such as Maven or SBT), but right now Gradle seems to me as the best choice for a building tool in Scala (that might change with CBT becoming more mature), so until IntelliJ further improve their support for Gradle configurations I'd keep delegating my tests to Gradle.

Anyhow, the task at hand is much clearer now. Thank you. I don't think it should be much of a problem implementing it, along with detecting changes in the project and re-generating the configuration (assuming Bloop is capable of detecting the changes when re-compiling).

[vigoo]

I did something very similar to the second phase integration a few years ago for Haskell, generating configuration for and invoking the stack compiler (https://github.com/prezi/gradle-haskell-plugin). I'm happy to help with this one.

[vigoo]

FYI I started working on this and have an initial "first stage" implementation.
https://github.com/scalacenter/bloop/compare/master...vigoo:gradle-support?expand=1

I will open a PR soon but first want to test it better and make a few things configurable.

[jvican]

@vigoo I am really impressed about your proof of concept. Especifically, the gradle integration you've added, that includes copying the gradle API and all so that we can test it from sbt. Looking forward to a pull request.

[jvican]

The gradle PR has been merged, instructions to use it are available on the website.

[dsilvasc]

jcenter has gradle-bloop 1.0.0+37-ad2ae774 for scala 2.11:

https://jcenter.bintray.com/ch/epfl/scala/gradle-bloop_2.11/

but nothing for 2.12:
https://jcenter.bintray.com/ch/epfl/scala/gradle-bloop_2.12/

and no 'gradle-bloop' in maven central:
http://repo1.maven.org/maven2/ch/epfl/scala/

[dsilvasc]

filed https://github.com/scalacenter/bloop/issues/606 for the missing artifacts

[timcharper]

@dsilvasc the artifact is named differently for some reason:

https://jcenter.bintray.com/ch/epfl/scala/gradlebloop212_2.12/