Moor: Using with built_value

[knaeckeKami]

It would be nice to be able to use DataClass-objects as members of built_value objects.
The default configuration of build_runner doesn't allow this, at the time the built_value classes are generated, the generated DataClass files are not visible.

It should be possible to do this with a build_runner configuration, but I couldn't figure out how to make it work.
If someone knows how to do this, it would be nice to add documentation for this, as it is a somewhat common use case in my opinion.

[simolus3]

The problem here is how source_gen builders (both moor and built_value) interact with each other. The builders will work like this (order of step 1 and 2 is not fixed): (edit: this is slightly wrong, see my next reply)

1. Moor generator creates a file called a.moor_generator.dart for the input file a.dart
2. built value creates a file called a.build_value.dart (or similar) from a.dart
3. The shared part builder merges these files and creates the final a.g.dart.

Builders will only be called on files that strictly end with .dart (particularly, not .g.dart or .moor_generator.dart), so neither builder will be called on any generated file. We could make the moor generator emit a standalone part file, so that a users setup would look like this:

// file called a.dart
part 'a_moor.dart'; // for the code generated by moor. May only have .dart ending
part 'a.g.dart'; // for code generated by other builders

Then, the build.yaml can be configured so that the built_value_generator will always be run after moor_generator, so it should be able to see the generated files at that point.

This would be a breaking change and I'd prefer to only turn this behavior on when specifically requested by a user. If I remember correctly we can't make the output file ending depend on some configuration files with build. We could have two builders in moor_generator: One that works like the existing one and is enabled by default and another that works like described above and is disabled by default. If that feature is needed the first builder could be disabled and the second one enabled.

I'm not sure if we're going to run into another issue with that setup, I'll try to test this tomorrow.

[knaeckeKami]

Cool!
Are you sure that it wouldn't be possible to do this with a build.yaml file in a project that is using both built_value and moor?
I thought, using "runs_before" to make sure moor runs before built_value and "build_to" : "source" might work. It seems like this option is there to configure dependencies between builders.

[simolus3]

My idea didn't work :/ I think my assumptions about the file endings was false, but the point still holds:

1. the first builder generates its .builder_a.part file
2. the second generates its .builder_b.part file, the final .g.dart file doesn't exist at this point
3. source_gen merges the two files together afterwards, so each builder can't see what the other one has been doing.

I tried a setup where I generate two part files (one for moor, one for all the other builders) and then run built_value after moor_generator:

1. moor generates a .moor.dart file, the source file has a part 'name.moor.dart' in it.
2. built_value generates a .built_value.part file, the file created by moor already exists at this point
3. source_gen basically renames .built_value.part to .g.dart as there is no other builder.

Sadly, it still complains about the type not being found :( It might be that the Dart analyzer isn't running between two builds so that built_value can't see the generated file yet. I've asked someone who ran into a similar problem before about this, if we don't find a solution I'll also ask someone on the Dart team. Surely it can't be that hard to compose builders, yet I can't find any documentation on it...

[jerryzhoujw]

What about to create a SerializerPlugin

[simolus3]

Thanks for the suggestion, but I'm not sure if generating a serializer plugin would help here. The problem is that all the classes moor generates aren't visible to other builders (including built_value_generator, but this would affect any other package generating code with the build system).
I've asked about this on the dart-lang/build gitter channel, I'll keep you updated when I get a response. That said, I don't think the authors of built_value can do much about this as appears to be a problem that is common to all builders.

I've seen a workaround that consists of putting code that uses moor and code that uses built_value into different files and then running the build_runner twice. That works, but it kind of feels wrong to use.

[davidmorgan]

I suggest opening an issue on package:build if you didn't get any answers from dart-lang/build. I'd be interested to know if they can point you to a good solution.

[simolus3]

Just opened https://github.com/dart-lang/build/issues/2389. I tried to find a minimal builder to reproduce this, it looks like a bug with the resolver because this only happens when the first builder also calls BuildStep.inputLibrary. If the first builder just generates its code without invoking the analyzer, there is no problem.

[knaeckeKami]

Wow, interesting! This shows how that this ecosystem is still pretty young, I can't believe that we're the first people stumbling on this...

[simolus3]

I was almost getting crazy trying to getting the builders lined up, but now I found a fairly small configuration that works. With that commit (on develop, it will be part of moor 2.4), you can use this build.yaml file:

targets:
  $default:
    builders:
      # don't make moor generate a .g.dart file, write a .moor.dart file instead
      moor_generator:
        enabled: false
      moor_generator|moor_generator_not_shared:
        enabled: true
      # we're running built_value later
      built_value_generator|built_value:
        enabled: false

  run_built_value:
    # run the $default target first, to ensure moor_generator is done
    dependencies: ['with_built_value']
    builders:
      # disable all builders (that are on by default), except for built_value
      moor_generator:
        enabled: false

An example of a moor data class being used in a Built object is here:
https://github.com/simolus3/moor/blob/cd9b7101b42e2992fa94caad4a80d680095e4ae8/extras/with_built_value/lib/database.dart#L8-L18

I'll update the documentation website with a proper example tomorrow.

[simolus3]

More explanations are available in the preview docs, those will go live with the next release. The feature is available on the beta branch as well now. I'm aiming for a release next week but I can't make any promises. Feel free to report any issues you run into with this.

[neckaros]

@simolus3 i tried the build.yaml that we can find in the doc:

targets:
  $default:
    builders:
      # disable the default generators, we'll only use the non-shared moor generator here
      auto_apply_builders: false
      moor_generator|moor_generator_not_shared:
        enabled: true
        # If needed, you can configure the builder like this:
        # options:
        #   skip_verification_code: true
        #   use_experimental_inference: true
      # This builder is necessary for moor-file preprocessing. You can disable it if you're not
      # using .moor files with type converters.
      moor_generator|preparing_builder:
        enabled: true

  run_built_value:
    dependencies: ['xxxxx']
    builders:
      # Disable moor builders. By default, those would run on each target
      moor_generator:
        enabled: false
      moor_generator|preparing_builder:
        enabled: false
      # we don't need to disable moor_generator_not_shared, because it's disabled by default

but i get this error:
Invalid argument(s): line 5, column 7 of build.yaml: Unsupported value for "builders". type 'bool' is not a subtype of type 'Map<dynamic, dynamic>' in type cast

Do we have to use a new build.yaml with latest build_runner (build_runner: ^1.11.0)?

[simolus3]

Oops, the auto_apply_builders should be in the target section, not under builders.

targets:
  $default:
    # disable the default generators, we'll only use the non-shared moor generator here
    auto_apply_builders: false
    builders:
      moor_generator|moor_generator_not_shared:
        enabled: true
        # If needed, you can configure the builder like this:
        # options:
        #   skip_verification_code: true
        #   use_experimental_inference: true
      # This builder is necessary for moor-file preprocessing. You can disable it if you're not
      # using .moor files with type converters.
      moor_generator|preparing_builder:
        enabled: true

  run_built_value:
    dependencies: ['xxxxx']
    builders:
      # Disable moor builders. By default, those would run on each target
      moor_generator:
        enabled: false
      moor_generator|preparing_builder:
        enabled: false
      # we don't need to disable moor_generator_not_shared, because it's disabled by default

I'll fix this in the docs, thanks for pointing it out!