Dart-code: Support debugging Flutter widget tests

[xster]

It'd be nice to get https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/commands/ide_config.dart to set up tasks.json such that when in a file that uses testWidgets from import package:flutter_test/flutter_test.dart, it could automatically invoke flutter test against that file and debug with breakpoints on cmd-shift-T.

[DanTup]

Possibly this is something for the flutter repo; though I do have some tasks about contributing task providers #499, #87. Probably we can also set a recommend extension in there too, so if someone runs that command for VS Code then opens VS Code but does not have Dart Code installed they'll get a "recommend extensions" prompt.

Testing is nearing the top of my list so if I can get some time to set it up to run tests for myself locally, it shouldn't be hard to then mirror into that script.

[iampawan]

+1

[DanTup]

flutter test has a --machine flag which talks JSON which I think we should ultimately use. However for now, since we don't have a good way to render the results I think we should just run the command in the same way that we do the Dart VM (eg. as a console app and connect to Observatory).

I got this working but it needed some possibly-fragile changes and also a bug fix (?) in Code

Changes involve:

• Returning a new entry point from getDebuggerExecutable, flutter_test_debug_entry.js - requires https://github.com/Microsoft/vscode/issues/45220 (I hard-coded for testing)

• Implementing a new sub-class of DartDebugSession named FlutterTestDebugSession that overrides spawnProcess to call flutter test in place of the Dart VM:

  protected spawnProcess(args: FlutterLaunchRequestArguments): any {
      const debug = !args.noDebug;
      let appArgs = ["test"];
      if (debug) {
          appArgs.push("--start-paused");
      }
      if (args.previewDart2) {
          appArgs.push("--preview-dart-2");
      }
      appArgs.push(this.sourceFile);
      if (args.args) {
          appArgs = appArgs.concat(args.args);
      }
  
      const process = child_process.spawn(this.args.flutterPath, appArgs, { cwd: args.cwd });
  
      return process;
  }
  

• Passing script uris as file paths (this feels like a bug and might bite us if we do this? - see https://github.com/dart-lang/sdk/issues/32500)
• Changing the Obseravtory port regex to not require Observatory listening on prefix (this also feels slightly fragile, just picking up any url - maybe we can read the text from the few lines before?)

[DanTup]

We should check if the JSON format is the same as this:

https://github.com/dart-lang/test/blob/master/doc/json_reporter.md

If so, it might be worth going the JSON route as it'll be re-usable for CLI and will become easier to build a UI on in the futuer.

[DanTup]

Got this kinda working:

However... There are some issues - mainly that we can't tell at debug-launch time that we need to use the test debugger (flutter test --machine is very different to flutter run --machine).

In order to fix this, we need some Code issues addressing:

• https://github.com/Microsoft/vscode/issues/45580 (fixed for v1.22, shipping in April)
• https://github.com/Microsoft/vscode/issues/45220 (in the backlog 😢)

I spent some effort trying to make a single debugger script that could delegate to the others, however it didn't seem to work - seems like when the debug client creates a DebugSession it must call a bunch of methods to set up the communication and they don't work if I created the instance myself. I've dumped the WIP on a branch and asked for info from MS about whether it's possible (I suspect they'll say it's a bad idea).

So, this probably is going to be next month at earliest. I'll leave it in this milestone to re-review the options towards end of month.

[DanTup]

Ok, the way this works for now is:

When you hit F5 in a flutter project, if there is no program specified in your launch.json (or you don't have one) and the active file is in the test folder, then it will be launched via flutter test. So, if you have a test file open and hit F5 it'll run that, otherwise it'll launch your app as usual.

I'm going to create a beta release of v2.11 in the next few days and would really appreciate some testing/feedback of this as there have been some significant changes to make it work (we now control which debug adapter is used and launch it in-process, whereas it used to be controlled by Code and launched in its own process).

[vogella]

Thanks @DanTup for this, every useful.