Dart-code: Test runner doesn't use experiments in analysis_options.yaml

[lukepighetti]

MCVE

lib/foo.dart

typedef Foo = String;

Foo foo(){
  return 'foo';
}

test/foo_test.dart

import 'package:dart_code_3243/foo.dart';
import 'package:test/test.dart';

void main() {
  test('foo', () {
    expect(foo(), equals('foo'));
  });
}

analysis_options.yaml

include: package:pedantic/analysis_options.yaml

analyzer:
  enable-experiment:
    - nonfunction-type-aliases

pubspec.yaml

name: dart_code_3243
description: An MCVE for Dart-Code/Dart-Code issue 3243.

environment:
  sdk: ">=2.13.0-0 <3.0.0"

dev_dependencies:
  pedantic: ^1.9.0
  test: ^1.16.8

Error message when clicking run tests button

Failed to load "test/foo_test.dart":
Unable to spawn isolate: lib/foo.dart:1:13: Error: Can't create typedef from non-function type.
typedef Foo = String;

Expected:

it uses analysis_options.yaml to enable experiments.

Request for clarification

Is this intended behavior? Is there any way to enable experiments across all of Dart-Code's features including running/debugging/analyzing/testing?

[DanTup]

I think this expected - the experiment is only read by the analyzer, not the VM when you run the code.

I don't know if there's a single way you can enable this for both - perhaps @leafpetersen or @mit-mit may know?

[lukepighetti]

A single point of enabling project wide experiments would be sweet.

[leafpetersen]

Yes that is correct. We don't want to make the compilers have to look for and read analysis_options.yaml, so I don't think we would go this route. The experiment flags are intentionally somewhat manual to use, but there has been some discussion of ways to make trying out experimental features and providing feedback easier (without giving up our ability to make breaking changes to experimental features). I think @jakemac53 may have had some ideas there that might be worth revisiting at some point.

[lukepighetti]

Thanks for weighing in, I'm going to close this issue since it's working as expected.

Some thoughts:

Perhaps analyzer/test/vm could read experiment flags from pubspec.yaml? That said, I don't think it necessarily makes sense to have this stuff in two places in the core tooling.

Maybe VSCode could have an opinionated "experiment flags" setting that automatically pipes it to all of the tooling.

There are cases when we need to pre-empt a feature that we know is going to production, ie nonfunction-type-aliases, with the intent being to simply remove the flag when it's in production without fiddling with the rest of the tooling stack.

{
  "dart.experiments": [
    "nonfunction-type-aliases"
  ]
}

This doesn't resolve CI/CD, but it would resolve development. Would be nice to find something that is an e2e solution.

[jakemac53]

I am definitely in favor of an integrated solution here. But yes there isn't anything for DartCode to do so closing this issue is I think the correct choice.

Fwiw my strawman approach for this would be to add an experiments field to the package_config.json file. This is the file intended to be consumed by tools, and the one they currently read today.

Pub would add support for filling that in, based on an enabled_experiments field (likely under the environment section).

You would be allowed to publish packages which enable experiments, but they would usually be ignored for version solving. They would only be allowed if the root package doing the version solve also enables all the experiments enabled by the version of the package. So it is essentially treated as an extra constraint on the version solve.

Note that pub should also impose strict sdk constraint requirements for anything enabling an experiment. It should only allow a single major version of the sdk, and not future sdks.

[lukepighetti]

Personally I wouldn't mind if Pub blocked packages with an experiment enabled. But everything else sounds great. One interesting thing is I have literally never seen or looked at a package_config.json file in my life.