Flatbuffers: Dart support

[isoos]

I'm wondering if there are plans for supporting Dart too (especially after the JS serialization is done, I imagine it wouldn't be that much different to do the codegen).

[aardappel]

I believe there has been some work on a Dart implementation, but it was never released. I don't know of anyone actively working on it, no.

[dnfield]

It looks to me like the Dart SDK implements at least some of it here:

https://github.com/dart-lang/sdk/blob/master/pkg/front_end/lib/src/base/flat_buffers.dart

@scheglov any good reason to not just split this out into its own package? I think this would be interesting to use particularly in Flutter projects

[asaarnak]

@dnfield Thanks for a comment, Google Search got me here. :)

Currently the official flatc schema compiler is not supporting dart code generation.

Seems only a format.fbs and format.dart files generated from idl.dart are using the flat_buffers.dart

Generation script:

1. format.fbs generate.dart#L377
2. format.dart generate.dart#L432

[dnfield]

I'm working on dart codegen.

[asaarnak]

Oh :) Then i was just investigateing interesting code.

[scheglov]

Yes, we use flat buffers in Dart, and like it.
Ideally yes, it would be a separate package.
Unfortunately we don't have time to do this and support it later :-(

[isoos]

@scheglov: How much work is needed to cover the spec and separate it into a package? Is the spec otherwise covered? I'd volunteer for some of that work.

[scheglov]

You're most welcome to use any existing flat buffers code for Dart. I honestly don't remember exact difference between the spec and implementation. I know we added one or two new things, like list of booleans.

[aardappel]

I'd generally recommend supporting it through the official compiler (flatc), as that'll make it more likely the implementation is correct, complete, and will have some future support. If anyone wants to work on a PR I'll happily review it.

[dnfield]

Yes, I'm working on implementing the code generation for flatc. However, it'd make sense for there to be a flat_buffers dart package (which would basically reuse the code implemented already) - similar to the flatbuffers npm package for JavaScript.

[aardappel]

@dnfield the npm package is generated from code in this repo, would it work similarly for Dart?

[dnfield]

I don't see why not.

[dnfield]

How should we handle the publication of flat_buffers.dart? I've had to make a few additions from what was in the front_end package (it was missing helpers for adding some typed data, like Int16/UInt16 and Float32s). I'm adding the up to date file to my brach of google/flatbuffers which has my start on Dart code gen.

Who owns, for example, publishing flatbuffers.js to NPM?

[aardappel]

I push to npm whenever I make a versioned release. If doing the same for Dart is just updating a version number and running a single command, I can do it as well :)

[binary132]

I am really interested in this! I have two questions:

1. can flatc support a plugin model for language extension, like protoc? Otherwise, I feel the reach will be limited to only support languages which flatc chooses to support.
2. can we please take care to avoid the platform lock-in pitfall which the Dart protoc plugin has fallen into where it only supports Unix-y platforms? mingw / cygwin are not a solution for me.

[dnfield]

@binary132 - flatc compiles with Visual Studio and it looks like there are windows based binary releases. That should address your second point.

However, to be fair, I'm more concerned with using this for Flutter than for dart2js - my testing is much more around that and I can't guarantee I haven't broken anything for dart2js (particularly around int64 handling).

As far as its reach, it's entirely possible to implement the spec in another language - in fact, the Dart SDK team did something close to that. However, it makes it more challenging to keep up with new features/changes.

[binary132]

I'm also keen on Flutter, but good options in the browser are limited. Dart makes a good case to me as a multi-platform client runtime, which I'm looking to use flatbuffers for. I'll try to find time to take a look at the browser side of this. I am entirely new to flatbuffers, but it looks intriguing (especially since capnproto has weak Javascript options.)

There's already a Javascript target (right?), so maybe in the short-term this could be done by proxying to that plus generating a Dart-native wrapper. 🤔

But then you're not dealing with a single dart target, you're looking at a dart-native and a dart-js target.

[aardappel]

flatc currently has no plugin system. On the positive side, any code generators part of flatc are pretty much guaranteed to be kept portable to all desktop platforms and compilers.

[ghost]

I am keen on using this with Flutter in 2 ways:

1. As a Flutter plugin on the mobiles. Flutter client <--> golang plugin
2. golang client <--> golang server

[ghost]

Is anyone using flat buffers with dart yet ?

Would be great to have their opinion.

[isoos]

@aardappel: anything to be done before merging #4676?

[aardappel]

Can't load that page (has been giving me GitHub's Unicorn error for hours now). I seem to remember we are ready to merge, if there are no further comments.

[dnfield]

It loads for me. Either way do you want me to just create a new pr?

[aardappel]

It's nicer for "history's sake" to merge the original PR, give me a few hrs. I've contacted github about it.

[aardappel]

It just loaded, so I merged! Congrats to all involved :)

Since this is still fresh, if people feel we need API/code improvements, now is the time for follow-up PRs.

[isoos]

@aardappel: did somebody publish the Dart package to pub.dartlang.org?
Also related: the README.md and the homepage could also include the information about support. Shall I file separate issues for them?

Almost forget: huge thanks for everybody involved!

[aardappel]

@isoos I haven't published anything. If I need to be the person to do that, I'd appreciate instruction on what to exactly upon a release.. I know there's a version to be update in the yamspec, and probably a script to run.

The docs on http://google.github.io/flatbuffers are updated manually (usually upon a release), not automatically following each PR.

[dnfield]

@aardappel I created a script here: https://github.com/google/flatbuffers/blob/master/dart/publish.sh to help with publishing.

If you want to change the version, it needs to be changed here: https://github.com/google/flatbuffers/blob/master/dart/pubspec.yaml (Currently at version: 1.9.0)

[dnfield]

You would need to have pub and dart installed.

[aardappel]

I ran the script and it succeeded.

It suggested:

* Author "Konstantin Scheglov" in pubspec.yaml should have an email address
  (e.g. "name <email>").
* Author "Paul Berry" in pubspec.yaml should have an email address
  (e.g. "name <email>").

[isoos]

Yay: https://pub.dartlang.org/packages/flat_buffers