Bloop: Bloop does not use the root of the project as working directory

[Baccata]

Relevant gitter conversation

Bloop follows SBT's behaviour when fork is enabled. SBT considers the submodule's folder as CWD when fork is enabled, but the root of the project when fork is disabled.

My personal opinion on the matter is that this behaviour makes little sense :

1. As @tpolecat points out, forking is disabled by default in SBT. I don't think many people are aware that enabling it has the effect of changing the working directory when running/testing programs. Therefore, bloop's default behaviour differs from SBT's default behaviour.

2. Intuitively, I personally expect that a JVM program should consider whatever folder I run it from to be the CWD, unless specified otherwise. If my main is println(os.pwd) using os-lib, and I run it from ~/foo, I'd expect to see /Users/myUserName/foo printed in the terminal, whether I run this program from bloop or from a packaged jar . Not following this behaviour makes it weird when developing CLI tools using bloop

3. macros that capture location information, such as the ones from sourcecode, are extremely useful for things like test-frameworks. In particular, it enables neat workflows in vscode when you click on an error in your terminal to jump directly to the relevant piece of code. However, bloop makes it harder for that workflow to work, by making it impossible to create a sensible relative path since the root's of the project isn't available from the runtime unless you are aware that you need to set it manually in your java options. As a result, bloop, which makes running tests one of its goals, gives me a worse UX than the build tool when I'm testing.

I hope that my examples show that this behaviour breaks the principle of least astonishment. Moreover, using SBT's behaviour as a justification for it goes against bloop being build-tool-agnostic, as the exact same behaviour impacts all users using alternative bloop-able build tools 😉

[olafurpg]

@Baccata thank you for this suggestion! I agree it's confusing when run/test in bloop uses a different directory from the sbt build when fork := false (which is the sbt default).

Since https://github.com/scalacenter/bloop/pull/1154, you can work around this behavior with the following settings

// build.sbt
javaOptions += s"-Duser.dir=${baseDirectory.in(ThisBuild).value}"

One thing we could do is change sbt-bloop to automatically add this -Duser.dir option when fork := false and preserve the current behavior when fork := true.

[Baccata]

One thing we could do is change sbt-bloop to automatically add this -Duser.dir option when fork := false and preserve the current behavior when fork := true.

That'd be an improvement, but it'd only address the (perceived) weirdness for the specific case of SBT though, and other build tools (mill) are subject to the same issue.

In particular, as the person who wrote the bloop-mill integration, I was unaware that this would be a consequence of setting the directory field in the "project" payload. In the absence of scaladoc, the meaning of that field would have been much clearer had it been called "workingDirectory". A non-doc comment in the source, to understand what the field in question does, would go a long way (same goes for the workspaceDir field. Its meaning seems obvious but how bloop uses it isn't)

Out of curiosity, is there a build-tool-agnostic rationale as to why the current behaviour makes sense ?

[olafurpg]

Other build tools can also use the user.dir system property so I don’t think it’s an sbt-only solution. The directory field is important for IntelliJ and should be unique from other Bloop projects (excluding the companion test project).

[Baccata]

Other build tools can also use the user.dir system property so I don’t think it’s an sbt-only solution

My point was more that since the behaviour makes little sense (to me, anyway), I find it somewhat unfair for the build tools integrations, or even users, to have the responsibility to work around it. That's also why I'd like to understand arguments for keeping it as it is (as opposed to changing it at the root).

NB : "we made that decision and now it's too late to change because users might depend on it" is a totally receivable argument 😄

The directory field is important for IntelliJ and should be unique

That is absolutely fair.

[olafurpg]

I think it’s a good idea to add a new optional “workingDirectory” field that defaults to “directory” when not present. It’s a better solution than user.dir because it’s cross-platform (works for JVM/JS/Native).

[jvican]

Thanks for starting the discussion on this @Baccata, I appreciate the time you took to write this out in such a detailed manner.

Bloop follows SBT's behaviour when fork is enabled.

I hope that my examples show that this behaviour breaks the principle of least astonishment. Moreover, using SBT's behaviour as a justification for it goes against bloop being build-tool-agnostic, as the exact same behaviour impacts all users using alternative bloop-able build tools

For context, Bloop also follows the way it works in Gradle and Maven where independent project directories are more of a first-class citizen because the build configuration lives right at the root of every subproject. Bazel has the same semantics. So I posit that users using these libraries with bloop will need to a mind shift before running an application with bloop.

The principle of least astonishment can be applied from different optics depending on what you're used to and what you're not to.

I think it’s a good idea to add a new optional “workingDirectory” field that defaults to “directory” when not present. It’s a better solution than user.dir because it’s cross-platform (works for JVM/JS/Native).

I think this is a good proposal.

After thinking a while about this, I think what makes the most sense is to populate workingDirectory by the origin build tool with their concrete semantics and avoid defining a common behavior across all bloop run usages. This is a way to can have our cake and eat it too. Tools such as Gradle, Maven and Bazel can use the semantics we use today and sbt can populate the project working directory with the root working directory.

This is a big change of the semantics of bloop run though, so I expect there will be breakage that will require manual user intervention. Given how intrusive manual intervention is, I'll make this change and prepare release for 2.0.0.

Don't despair, I don't intend to make 2.0.0 so far away.

[olafurpg]

This doesn't need to be a breaking change and I'm still not sure it's necessary to introduce a new workingDirectory field. It should be possible to fix this with a few line change in sbt-bloop to set the -Duser.dir system property based on fork.value.

[jvican]

That's a good callout, yeah, we don't need this to be a breaking change. I'll fix this in the build tools we maintain, @Baccata could you fix this in the mill bloop integration?

[Baccata]

Thanks a lot for providing context.

@Baccata could you fix this in the mill bloop integration?

@jvican can do. What about JS/Native platforms though ? Will bloop respect -D.user.dir for them too ?

[olafurpg]

@Baccata currently it doesn't work that way for Native/JS projects but I think it could, the logic is here https://github.com/scalacenter/bloop/blob/a2040035b6a9ec795414c222ca1940e6b78caa51/frontend/src/main/scala/bloop/data/Project.scala#L73

[olafurpg]

I think it would be an acceptable solution to first make it work for JVM projects because it's low-effort and open up a followup ticket to solve the problem for JS/Native.

[Baccata]

Agreed.