Build: How should we deal with flutter and "dart:ui" in generators?

[davidmorgan]

Running the built_value generator on flutter code currently spits out a severe error:

[SEVERE] Instance of 'PartBuilder' on dartsugar|lib/models/module_data.dart: Unable to resolve asset ID for "dart:ui"

It still works, but I wonder if 1) there is potential for things to break down the line, in which case we should make sure this doesn't happen; and 2) there's a way to do it without getting the warning. Any ideas please?

See: https://github.com/google/built_value.dart/issues/282

[jakemac53]

This seems like a bug, it looks like you are hitting this line, which has an assert above it that the scheme != dart, so in checked mode this would throw.

[natebosch]

This could be tricky. The SDK we use while generating the code doesn't have dart:ui so it probably can't resolve any of it. We'll probably have to add the ability to point to a different SDK or something for the Resolver?

[matanlurey]

Yeah, we'll need a way to use Flutter's Dart SDK, not the same as which dart.

I'm making this M1 because it doesn't prevent the initial MVP for the web.

[davidmorgan]

Thanks. Yes, I don't think this is blocking right now.

[matanlurey]

I thought about this a bit, this is tricky, because the Flutter Dart _tool_ SDK isn't the same as the Flutter Dart _engine_ SDK; we need the second one if you want to be able to resolve dart:ui.

[s-bauer]

Any update on this? It's a problem because the reflectable package is also unable to correctly handle types like Color from the dart:ui package

[eernstg]

@matanlurey, concerning

the Flutter Dart tool SDK isn't the same as the Flutter Dart engine SDK;
we need the second one if you want to be able to resolve dart:ui.

it would be really nice to find a solution, but you don't sound like it's easy. But why isn't it a problem for anyone else that we have that difference?

Here's a naive idea:

Presumably the problem arises because the tool SDK just can't include 'dart:ui' (that is, 'ui.dart' and its parts, and maybe other things that it depends on). But do you think it would be a viable approach to include a completely unimplemented version of 'dart:ui' sources in the tool SDK? It might be a problem to maintain it, unless we can produce such an unimplemented variant of the code mechanically (Perl? ;-).

WDYT?

[matanlurey]

But do you think it would be a viable approach to include a completely unimplemented version of 'dart:ui' sources in the tool SDK?

Yes, that would be quite impossible. It would require syncing back the Flutter engine to the main-line SDK, or constantly being behind - i.e. almost every Flutter engine update would break compatibility. I would not want to sign up my worst enemy for this process.

[eernstg]

Considering the explanation from @matanlurey once more:

the Flutter Dart _tool_ SDK isn't the same as the Flutter Dart _engine_ SDK;
we need the second one if you want to be able to resolve dart:ui.

So, if the tool SDK is used to run the analyzer, compiler, etc. when working on Flutter related software (e.g., via flutter run), and those tools will check the given Dart source relative to the engine SDK because they are invoked with a suitable --dart-sdk argument (or whichever option each tool wants for that purpose), then we might actually be able to solve this problem by making sure that the build package and similar invokers of code generation engines pass a suitable option specifying which SDK path to use. Right?

[zoechi]

the Flutter Dart tool SDK isn't the same as the Flutter Dart engine SDK;
we need the second one if you want to be able to resolve dart:ui.

Is this still true?
I think to remember that the Flutter team wanted to get rid of the tool SDK to reduce download size. Not sure if this actually happened.
@Hixie ?

[Hixie]

We now build our own version of the Dart SDK, but we still end up with two variants of Dart (both at the same version), one for running command line tools (compiled for x86 on Windows, Mac, Linux) and one for running Flutter applications (compiled for ARM on iOS and Android). Actually there's even more variants than that (debug mode, release mode, ARM32, ARM64, etc).

[davidmorgan]

This is showing up as a concrete issue over on built_value: people can't use dart:ui types, in this case Color, in classes that use codegen:

https://github.com/google/built_value.dart/issues/513

[jakemac53]

Probably the easiest fix here would be to allow a custom SDK to be provided somehow, although that doesn't allow multi-platform packages still, and could even cause weird errors in dependencies that are building to cache and have code that requires dart:html (even if you don't use that code).

[jakemac53]

Once we complete the analysis driver migration we can look into using a summary based resolver.

Assuming we can get that to work reasonably within build_runner - we could also potentially make sure that the resolver we provide is a platform specific one.

We already have a notion of platforms in the modules, so we should be able to choose a compatible SDK summary for the platform.

[tarobins]

Any update on this? Ran into it while trying to use a built_value containing an Offset.

[jakemac53]

No updates at this time. We have also moved away from analyzer summaries generally (in the build system), which makes the summary based resolver approach a lot more expensive if we wanted to go that route.

cc @stereotype441 for fyi

[eernstg]

Here's another fyi: The issue described here is very likely the same that causes the problems reported in dart-lang/reflectable#172. So package:reflectable is also waiting for a solution.

[jakemac53]

Note that we are now creating our own SDK summary in build_resolvers, so we could add some special handling presumably to recognize flutter packages and create a different summary in that case.

[lesnitsky]

@jakemac53 could you please share more details about build_resolvers? Is this already working with dart build?

[eernstg]

Yes, please! ;-)

[jakemac53]

The AnalyzerResolvers constructor now takes an optional function that returns a Future<String> which is the path to a summary it wants you to use.

Here is the default implementation of that which builds the summary and does some basic invalidation based on analyzer/sdk version.

I think it would be reasonable for the default implementation to also support flutter. I don't know exactly what that would look like, it is just one line to summarize the dart sdk. cc @scheglov do you know?

[scheglov]

I don't understand what you are asking about. Yes, SummaryBuilder can be used to build summary of Dart SDK. If it happens to be the Dart SDK from Flutter, with dart:ui, then it will get into summary. Give it the path to the SDK, and it will return the summary bytes.

[jakemac53]

I don't understand what you are asking about. Yes, SummaryBuilder can be used to build summary of Dart SDK. If it happens to be the Dart SDK from Flutter, with dart:ui, then it will get into summary. Give it the path to the SDK, and it will return the summary bytes.

The dart:ui implementations don't live under the dart sdk portion of the flutter sdk afaik (otherwise this would just work).

[scheglov]

Ah, well, then it will not work.
How does dart:ui get into Dart SDK?
I remember something vaguely about embedders, but I don't know how it exactly works to propose something that I would be sure to be the right solution.

[jakemac53]

https://github.com/dart-lang/build/pull/2501 should solve this, if somebody can try it out that would be cool, I will be making sure it works tomorrow. You can add these dependency overrides to your pubspec.yaml:

dependency_overrides:
  build_resolvers:
    git:
      url: https://github.com/dart-lang/build.git
      ref: support-flutter
      path: build_resolvers

[jakemac53]

I have confirmed this resolves things. You do have to run some flutter setup before it works, it lazily downloads the engine which contains the necessary files to resolve dart:ui. I added the required setup to an error message, but for most users this shouldn't be an issue.

[shyndman]

Thanks Jake!

[jakemac53]

This has now been released as build_resolvers 1.2.0 - you may need to rm -rf .dart_tool/build_resolvers to see the effect of the change.

[jakemac53]

Ok I just published 1.2.1 as well which will automatically invalidate the summary file as one would expect, so you won't need to manually delete it.