Metals: Run Scala Unit Tests from Visual Studio Code

[oharboe]

Is your feature request related to a problem? Please describe.

No

Describe the solution you'd like

I'd like to be able to right click on a .scala file to run the unit-tests in that .scala file.

Describe alternatives you've considered

Run from command line: sbt "test:testOnly DressRehearsal

Additional context

Search terms:

scala unit test

[tgodzik]

We are actually working on that at the moment, we'll keep you posted.

[oharboe]

Great! I've been using Eclipse up until now, but today I had occasion to switch. Eclipse was throwing exceptions and I've been ogling my colleagues C/C++ Visual Studio Code setup. So far, it has been a very pleasent experience. I'm a bit unused to the darker color themes, compared to Eclipse, so my eyes will need to a few days to adapt.

I'm actually using Scala for Chisel, a hardware description DSL: https://chisel.eecs.berkeley.edu/

[olafurpg]

@oharboe one workaround for now is to install the Bloop command-line interface https://scalacenter.github.io/bloop/ With the upcoming 1.3.0 Bloop release, you will be able to run bloop test ... from a terminal using the same compilation as in Metals. Compilation requests from Metals and the terminal will be automatically de-duplicated.

[oharboe]

@olafurpg Bloop sounds like a really neat solution. I'm a bit daunted by the installation process and number of moving parts.

A problem I immediately ran into was that I have a bug in my code which causes a compilation problem in bloop(nothing interesting; just some incorrect assumptions about paths), however the output from the bloop test command shows me a bunch of stacktraces(relating to the TCP/IP communciation with the bloop server), except the one that I need to find out where my application is broken.

I now realize that I not only need to run my unit-tests, I also need to be able to debug them in Visual Studio Code.

Silly questioon: is it possible to run the "bloop test" in the Visual Studio Code debugger?

[olafurpg]

relating to the TCP/IP communciation with the bloop server

What version of Bloop are you running and how did you install it? There was a new 1.3.0-RC1 release yesterday.

is it possible to run the "bloop test" in the Visual Studio Code debugger?

No silly questions ;) This is not possible currently but it will be possible eventually after the work @marek1840 is doing in https://github.com/scalameta/metals/issues/652.

[oharboe]

@olafurpg I installed 1.3.0-RC1. The compilation problem is not a problem in bloop, it's a bug in my code: a dirty hack to deal with dependencies on a resource file. It happens to work in Eclipse and sbt, but that's just because I've added some hacks. I'll clean that out.

I think I'll back off slightly from the bleeding edge here and use the metals editor for now.

[olafurpg]

Metals automatically connects to your installed Bloop server so you may want to stop it with bloop exit or brew services stop bloop (if you're on macOS). Metals starts an embedded bloop server when it's not installed on the computer.

[jvican]

@oharboe Off-topic here, but I'd love if you can drop by our Gitter channel https://gitter.im/scalacenter/bloop and walk me through all the steps you did to run into those TCP/IP connections, I've never seen them.

[oharboe]

@jvican I looked at the gitter thing, but it wanted me to sign in with github and I have no idea what that entails. It seemed to ask for more access than it needed, so until I understand that I'm putting a pin in that one.

In regards to the error message, the TCP/IP stack traces was just the context, the error was simply a compilation error.

You should be able to repeat the same by injecting any compilation error (a syntax error?).

[jvican]

In regards to the error message, the TCP/IP stack traces was just the context, the error was simply a compilation error.

Getting TCP/IP stack traces when there is any compilation error is definitely not the expected behavior. Could you try to reproduce it in a small project and send it my way? I'd like to get to the bottom of this error, I've never heard of anyone reporting this bug.

[oharboe]

@jvican Here's the message I got:

error.txt

[jvican]

Those errors are completely normal, the nailgun implementation produces them but they are benign and pose no problems to the correct behavior of bloop... Unless you are looking for something specific, you should completely ignore bloop server logs and instead focus on the logs you get in the clients (the CLI or Metals).

I'll try to figure out a way we can solve this upstream to avoid this kind of misunderstandings, I agree that the fact that users should not care about those errors is not clear at all.

Btw, it is rare that users run bloop server themselves, you would usually set up a system service to run it in the background. The installation guide elaborates further on this point: https://scalacenter.github.io/bloop/setup#universal

[oharboe]

@jvican The problem is that the server messages completely drown out the underlying error and that the underlying exception trace has been chopped off at it's knees with the "...": I can't tell where in my code it happens.

In regards to running bloop server manually: I was just trying to find out if bloop server could help in my case, so I didn't install it. For now it seems like it's easier to use sbt directly from a VSCode terminal to run my unit-tests.

I'm using Ubuntu, so I was hoping an "apt-get" would do the trick to install bloop. I'm unfamiliar with "nix-channel".

[jvican]

The problem is that the server messages completely drown out the underlying error and that the underlying exception trace has been chopped off at it's knees with the "...": I can't tell where in my code it happens.

Sure, but that's just a side effect of running the bloop server in the background of a terminal with &. Anyways, I'm happy this is not a more serious issue.

For now, it seems like it's easier to use sbt directly from a VSCode terminal to run my unit-tests.

Yes, it's easier. I nevertheless recommend using bloop because:

1. It keeps your compiler hotter for longer than Metals does, which means a faster development workflow. Metals throws away hot compilers every time a "Reload window" action is run or you kill VS Code.
2. It could potentially be faster at running tests, give it a try and see how snappy does bloop test $YOUR_PROJECT --watch feel.

I'm using Ubuntu, so I was hoping an "apt-get" would do the trick to install bloop

Maybe we can add some Ubuntu/Debian package in the future. There is however a generic installation via curl that would work out-of-the-box for your system.

Anyhow, I'm not commenting anymore in this ticket, I don't want to clutter it with off-topic comments. Let's keep it focused on testing :smile:

[oharboe]

@jvican I'll try with bloop again once I have fixed the bugs in my code. Thanks!

[oharboe]

I'm running sbt "test:testOnly DressRehearsal" from within Visual Studio Code, which does the trick for me.

My Scala project is actually a Chisel(a DSL for generating Verilog) project, which means that the amount of Scala code is tiny.

My concern is cognitive load(learning new tools, having to deal with setup, strange but harmless warnings messages, etc.) outside of the Chisel domain.

Being able to simply right click on a Scala Unit test and have the test results neatly presented and easy to navigate would help ease the cognitive load.

[jvican]

We're working on adding support for that. There's an open PR right now in Metals https://github.com/scalameta/metals/pull/942 after the appropriate changes we did in Bloop to enable that workflow. I suspect it'll be ready soon.

[oharboe]

@jvican Thanks! I'm impressed by the quality, performance and development momentum of Scala Metals, it really is a step up from Eclipse.

I wrote a note in this PR that I would try out bloop, which I never followed up on as invoking sbt directly from the VSCode terminal is easier than setting up bloop. The build performance advantage of bloop is not a big enough advantage to give up the convenience and fluidity of working only from within Visual Studio Code for the Chisel projects I work on.

[jvican]

Metals is using Bloop under the hood even without you doing anything. Installing it as a server gives you the performance advantages (which will be soon automatically enabled) for free as well as access to the CLI, which you can invoke from your VS Code terminal with bloop test MY_PROJECT.

[oharboe]

@jvican Gotit. Yes: I notice that Scala Metals is compiling blazingly fast, which makes editing and intellisense super-fast. I'm looking forward to dropping the invocation of sbt directly from the VSCode terminal from my workflow.

Thanks!

[olafurpg]

The latest 0.7.6+133-1dfcd90c-SNAPSHOT release from the current master supports running tests and main functions from VS Code. This functionality is still experimental, early feedback is welcome before we release the next stable version. Please open separate issues if you encounter problems with the latest SNAPSHOT

[FloWi]

The latest 0.7.6+133-1dfcd90c-SNAPSHOT release from the current master supports running tests and main functions from VS Code. This functionality is still experimental, early feedback is welcome before we release the next stable version. Please open separate issues if you encounter problems with the latest SNAPSHOT

This is an awesome feature, thanks for implementing it, @olafurpg !
Makes my life way easier!

[oharboe]

This is a great feature!

Is there a way to run a test within a suite?

For instance, a ThingAMaBobTest class might contain the test "work correctly", and I'd like to be able to debug that one test specifically and not all the tests in ThingAMaBobTests

sbt "test:testOnly *ThingAMaBobTests* -- -z \"works correctly\""

[mzarnowski]

Is there a way to run a test within a suite?

Unfortunatelly no, we are currently limited by the lack of a specific endpoint in the bsp. There is one for the test _classes_, but sadly nothing for the test _cases_ :disappointed:

[olafurpg]

That feature would be lovely to support but it requires custom integration per test framework (scalatest, JUnit, utest,...).

One feature we might be missing in bsp is to know what test framework is associated with each test class but otherwise the bulk of the missing work lies on the metals side to translate source code or tests into test framework arguments.

[olafurpg]

We should open a separate ticket to continue the discussion and we may need separate tickets per test frameworks (even separate tickets per testing syntax for example in scalatest, FunSuite, FlatSpec, etc)

[oharboe]

These names are not known until runtime, they cant be determined
statically, so how about asking the users with a dialog or I guess a
launch.json file in VSCode....

ons. 12. feb. 2020, 23:51 skrev Ólafur Páll Geirsson <
[email protected]>:

That feature would be lovely to support but it requires custom integration
per test framework (scalatest, JUnit, utest,...).

One feature we might be missing in bsp is to know what test framework is
associated with each test class but otherwise the bulk of the missing work
lies on the metals side to translate source code or tests into test
framework arguments.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/scalameta/metals/issues/1001?email_source=notifications&email_token=AAVLJZRLDX7MMHKOBJ4EM6LRCR4MHA5CNFSM4JCEEQX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELSWENA#issuecomment-585458228,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAVLJZRNEBLML4RL2VR6ZBDRCR4MHANCNFSM4JCEEQXQ
.