Sdk: Expose additional Dart APIs through ffi

[blagoev]

An issue from the MongoDb Realm team:

We got a lot of requests for creating a Flutter SDK for Realm mobile database (https://github.com/realm/realm-object-server/issues/55 now has over a 1000 upvotes) and initially made a prototype with FFI. Unfortunately, our progress is blocked for over a year now on https://github.com/dart-lang/sdk/issues/37022 and https://github.com/dart-lang/sdk/issues/35770.

We started to look at an alternative route and found one, but it would require adding official support for a set of the Dart VM API’s in Flutter.

Realm JS SDK is designed as a Node.js native extension with an abstraction layer that abstracts away the JavaScript VM and allows for targeting different JavaScript VMs on various platforms we support.

The similarity between the Dart VM APIs and the JavaScript VM APIs allows us to reuse a major part of Realm database code that is already tested and provide only a platform specific layer for Dart and Flutter. Moreover this approach would be superior to calling Realm through the FFI mechanism.

We made a POC with a custom built Flutter engine dynamic library which exposes a set of the Dart VM API’s and allows us to reuse a major part of Realm JavaScript engine abstraction layer with the Dart and Flutter platforms. We achieved it by exposing some of the Dart API’s from the libflutter.so library by editing the https://github.com/flutter/engine/blob/master/shell/platform/android/android_exports.lst.

In the light of the recent changes https://dart-review.googlesource.com/c/sdk/+/145592, we would like to request additional Dart APIs exposed officially so it can be consumed in Flutter through the Dart ffi. This will unblock us on delivering Flutter support and potentially will also enable other Node.js native extensions built on N-API and V8 API’s to be ported to Flutter and Dart in a straightforward manner, which would be beneficial to the Flutter and Dart community as a whole.

Here is a PR with the changes in question: https://github.com/dart-lang/sdk/pull/42931

cheers,
blagoev

[mraleph]

Unfortunately, our progress is blocked for over a year now on #37022 and #35770.

I think both of the issues now have their own workarounds (for asynchronous case there are send ports and for finalizers there are weak handles).

Are you still blocked even given those work-arounds?

we would like to request additional Dart APIs exposed officially so it can be consumed in Flutter through the Dart ffi

Unfortunately I don't think we want to do that. The set of Dart APIs exposes through FFI is carefully curated and we don't want to expose anything that is not absolutely necessary - I don't think we want to expose methods that you have added through FFI.

[ctrysbita]

I think this is useful.
Currently we have no way to access the data inside Dart_Handle if we pass an object as Handle to C through FFI. Then the only usage of Dart_Handle in FFI is to run the finalizer?

Allocate memory in C heap and copy the whole object (like large images) is inefficient under some critical situation. If we can access the internal pointers for reading these large object, the performace will be significantly better.

[mraleph]

Currently we have no way to access the data inside Dart_Handle if we pass an object as Handle to C through FFI. Then the only usage of Dart_Handle in FFI is to run the finalizer?

Yes, more or less. To attach finalizer or pass it back to Dart.

Allocate memory in C heap and copy the whole object (like large images) is inefficient under some critical situation. If we can access the internal pointers for reading these large object, the performace will be significantly better.

Could you expand your request with some code samples? I am mainly interested in how your data (large image) gets into the Dart heap in the first place and how it is represented there.

My current interpretation is that you are asking for a zero-copy way to access typed data lists from native? I think this is a valid request - we have always planned to include some way to automatically unwrap typed data into a raw pointer when transferring to native but we simply did not get to implement this. However some there are some challenges around it, which is worth outlining here. I guess what you want to achieve is this:

// Dart side
final list = Uint8List(N);

useList(list, list.length);

// C side
void useList(uint8_t* data, size_t length) {
}

and you want list to be passed without copying. For builtin typed list types this can be achieved by introducing a notion of leaf FFI calls. Such calls are not considered a GC safepoint: meaning that GC can no longer run while you are inside useList. This implies two things: useList should be a relatively fast operation and it can't call back into Dart. Note that data would only be valid for the duration of the useList call, e.g. something like this would be invalid:

void useList(uint8_t* data, size_t length) {
  // store for later use or pass to another thread
  smth->data_ = data;
}

If you want to support such use cases then data naturally needs to be allocated on the C heap to allow C side to use it without synchronising with Dart VM.

You actually have a way to kinda achieve that:

// Dart side
final memory = allocate<Uint8>(count: N);
final list = memory.asTypedList(N);

// Load image into `list`

useList(memory, list.length);

// C side
void useList(int8_t* data, size_t length) {
}

As you can see there are some possibilities - but we need to know more about your concrete use case.

[ctrysbita]

Yes, I want to archive zero-copy data transfer from Dart to C through FFI. Currently if we want to pass an image to C layer for processing, all data have to be copied into C heap.

Pointer<Uint8> memSpace = allocate<Uint8>(count: N);
var list = memSpace.asTypedList(N);

for (var  i = 0; i < N; i++) {
  // When N become larger, this operation takes too much time (especially under debug mode)
  list[i] = image[i]; 
}

// Pass Pointer to C.

And we have to free the C memory after all. This kind of memory duplication is not necessary since we are performing read only operation on these memory space or we want to simply apply filters that won't change the size of it.

I think the simplest way to archive this is to expose more dart VM api to FFI. We can aquire internal pointers using Dart_TypedDataAcquireData and persist the object using PersistentHandle. Like GetByteArrayElements and GlobalReference in JNI.

[mraleph]

@ctrysbita

I think the simplest way to archive this is to expose more dart VM api to FFI. We can acquire internal pointers using Dart_TypedDataAcquireData and persist the object using PersistentHandle

I would like to know a bit more about your API. Is the API you are passing the image data to synchronous?

Is my understanding correct that you want to write something like this:

final image = /* somehow load image data */

doSomething(image);  // your API

Here do you expect that doSomething processes the image and returns (after which point it does not need access to image any longer) or do you expect it to perform processing asynchronously in background?

I want to make sure that you realise that in general you can't achieve asynchronous/background processing neither through C API nor FFI unless you copy your data out of Dart heap. You can't access Dart handles outside of Dart mutator thread and when you call Dart_TypedDataAcquireData you effectively block GC from happening until you release it and once you release it you should not use the inner pointer you have gotten through acquire because it might become invalid. This for example means that you can't call Dart_TypedDataAcquireData to get a pointer into typed data and then pass that pointer to another thread to do background processing of typed data contents.

If these limitations are known to you and you are fine with them - then I think you should also be fine with the FFI extension that I have mentioned above: we do want to allow automatic unwrapping of typed data when it is passed to _leaf_ FFI calls. This would work as if there was a pair of typed data acquire/release operations around the FFI call itself.

I think, however, there is another question that we should ask here: why does image data end up in the Dart heap to begin with? Maybe this is the place where things should be designed differently.

[ctrysbita]

Yes, I know. Automatically unwrap the typed data is also acceptable. I mean exposing VM apis may be easier to implement and it gives more control to developers.

And... When will the automatic unwrapping become available ? :D