Go-flutter: How can I use golang on Android or iOS?

[chengxuncc]

I want to write an application on all platforms, it's a perfect solution working with go-flutter on desktop platforms. But I wonder how can I write apps efficiently on Android or iOS, I had seen go mobile or go bind, it seems I need to write code on Java/Kotlin or Swift, which is called by Dart and pass to Go API. I think there must be some ways simpler to call Go API in flutter.

[pchampio]

I have an idea on that subject, I'll describe it when I'm back from vacation.

Currently there no easy way to address this use-case.

[geoah]

I'd be interested in this as well. I've just moved to go-flutter-desktop and so far I'm really happy with the results. Being able to use go-flutter for ios/android would be an amazing endgame.

[gedw99]

Use gomobile is what I do.

[geoah]

@gedw99 are you using gomobile together with go-flutter-desktop?
If so could you please share how you are doing so?

[gedw99]

My requirements are different from others but happy to share and get critical feedback.

All I do is delineate between the flutter process and the golang process.
The many flutter apps are pure flutter running in the standard foreground process and the singleton golang process is running on a background process.

A couple of nasty aspects depending on what your tradeoffs are.

• the interop between the foreground and background adds latency.
• the manifest for deployment and signing and update is more complex

However for me the tradeoffs work for what my goals are:

• interoperability because it's just grpc with flatbuffers serialisation.
• no dart ffi And that risk.
• no need to use platform channels and take the Dev and runtime perf hit of context switching to java, swift etc.
• codegen is very standard stuff for background and client proxy and very well supported in both languages as well as rust etc
• security. Golang is doing advanced security / networking and you have less attack surface.
• onboarding of a user into the app gets the keys used for encryption out of bounds and into the flutter and golang processes.
• the golang process is a singleton and provides all data to the foreground flutter apps. So any data shared between two flutter apps stays in sync automatically. There is only one version of the ttruth on the device.
• security at rest is tighter by keeping it awAy from the flutter foreground processes
• desktop sharing between my apps and clipboard synchronization no longer leaks to the OS because we are not using it.
• multimedia is less risky project wise. We can swap to using Google desktop or rust-flutter when we need it and just use the same grpc / flatbuffers approach
• heavy bAckground processing is Always at the background layer.
• user experience is better because the background app is running doing pub sub and crdt style synchronization with other peers and servers and so when the user opens the app they have no delay in getting synchronised with the network. They already are.
• security related to notification locAlly but over the network to wake up a flutter app. The background app is the only way in from outside and I don't have to leak to Apple and Google's push servers. If I have Tobi can push just a ling through them to wake up the background and then the background will get the real message from my server.

The negatives ?

Your exposures to the shifting winds of what apple and Google allow for background services. But the advanced apps like Skype ( poor choice of advanced ) and many others use this technique and so it can never be blocked in all practicality

• flutter apps are in charge of their domain model. Golang singleton is a dumb pub sub topic based message queue. This forces me to keep the domain model and the message queue decoupled architecturally. As well as less experienced Devs tempted to break this contract. The flutters app can update to different versions and cope with it fine due to the schema evolution of flatbuffers just like protobuffs
• the flutter apps can store their data how they like and offers a cache to avoid hitting the background process. The background service just pushed updates to the subscribed flutter apps as messages come in.
• many background apps can be used. Rust is much better at some things like search indexing then golang. You can be flexible and just rely on grpc.

• it requires more effort but in other ways less effort.

• flat buffers support is a bit shaky and it's a really getting good eyeballs from Devs more and more. I use flatbuffers for everything P2P, C2S and S2S and am happy to have a bit higher complexity in return for much lower cost in running infrastructure and better UX.

  Protobufs are a great IDL to work from. You can use that IDL for flat buffers too these days. For flutter web from the IDL can be reused for that codegen.
  There is very little codegen though because it's just message queue semantics

[pchampio]

I had seen go mobile or go bind, it seems I need to write code on Java/Kotlin or Swift, which is called by Dart and pass to Go API. I think there must be some ways simpler to call Go API in flutter.

Projet like https://github.com/adieu/flutter go aims to help you in this task.

I would like to have similar features in go-flutter, here is 2 possibilities implementation:

• dart -> classic MethodChannel -> a thin java/swift (like what adieu has done) -> a small golang dispatcher -> your Golang plugin (exact same plug-in interface).
  Both the java/swift, and the golang dispatcher can me generated.
• dart -> a custom MethodChannel -> Dart FFI -> a small golang dispatcher -> your Golang plugin.

I like the second approach the most, but FFI on Android/iOS is still a open question for me.
Ref: https://github.com/go-flutter-desktop/go-flutter/issues/58

[gedw99]

Here is a working example of using GRPC between golang / gomobile and flutter.

The go code that is using grpc and exposes a GRPC client to flutter
https://github.com/textileio/go-threads/blob/master/api/pb/dart/pubspec.yaml

The flutter code that consumes the GRPC exposed by the golang code
https://github.com/textileio/dart-threads-client/blob/master/pubspec.yaml

---

The golang code is cross compiled using gomobile.

Here is an example: https://github.com/textileio/grpc-ipfs-lite/blob/master/Makefile

They use a golang lib to search for a freeport on the mobile when it starts up.

Thats basically everything.

So once that "plumbing" is in place you can do do things like:
https://github.com/textileio/go-threads#libraries

[chengxuncc]

Here is a working example of using GRPC between golang / gomobile and flutter.

It seem GRPC requires more work to be done, does it require local network? local network need to be concerned sometime and it's quite boring to define protobuf, not that directly calling function. I think Dart FFI is a better choice, I will try it first on mobile platforms.

[chengxuncc]

I tried Dart FFI but I don't known how to pass String type via Dart FFI. I have also taken a look at https://github.com/adieu/flutter_go/, it's great but won't work with latest flutter.
I think jsonrpc 2.0 is a good start, comparing to grpc, jsonrpc is simpler. https://github.com/adieu/flutter_go/ is using golang standard library net/rpc, which is not support mutual rpc. I consider reimplementing it by using jsonrpc 2.0 library like http://github.com/ethereum/go-ethereum/rpc.

[crossle]

@chengxuncc https://gist.github.com/sjindel-google/b88c964eb260e09280e588c41c6af3e5

[chengxuncc]

@crossle Thanks, but I don't understand why it didn't work without parameter.
main.go: go build -i -buildmode=c-shared -o ffi.dll

package main

import (
    "C"
    "fmt"
)

func main() {
}

//export Hello
func Hello() string {
    fmt.Println("called from ffi")
    return "Hello from golang via ffi"
}

ffi.dart: dart ffi.dart

import "dart:convert";
import "dart:ffi";
import 'package:ffi/ffi.dart';

class GoString extends Struct {
  Pointer<Uint8> string;

  @IntPtr()
  int length;

  @override
  String toString() {
    List<int> units = [];
    for (int i = 0; i < length; ++i) {
      units.add(string.elementAt(i).value);
    }
    return Utf8Decoder().convert(units);
  }

  static Pointer<GoString> fromString(String string) {
    List<int> units = Utf8Encoder().convert(string);
    final ptr = allocate<Uint8>(count: units.length);
    for (int i = 0; i < units.length; ++i) {
      ptr.elementAt(i).value = units[i];
    }
    final GoString str = allocate<GoString>().ref;
    str.length = units.length;
    str.string = ptr;
    return str.addressOf;
  }
}

typedef StringPtr = Pointer<GoString> Function(Pointer<GoString>);

void main() {
  final lib = DynamicLibrary.open('ffi.dll');
  final StringPtr Hello = lib.lookupFunction<StringPtr, StringPtr>("Hello");
  print(Hello(GoString.fromString("here is dart")).ref.toString());
}

It worked:

called from ffi
Hello from golang via ffi

Process finished with exit code 0

Then I modified ffi.dart:

typedef StringPtr = Pointer<GoString> Function();

void main() {
  final lib = DynamicLibrary.open('ffi.dll');
  final StringPtr Hello = lib.lookupFunction<StringPtr, StringPtr>("Hello");
  print(Hello().ref.toString());
}

It didn't work:

called from ffi

Process finished with exit code -1073741819 (0xC0000005)

environment:

dependencies:
  ffi: ^0.1.3
Dart VM version: 2.7.0
go version:1.13.6

[crossle]

ffi only support param struct pointer, not support return ...

[chengxuncc]

But it did returned.

It worked:

called from ffi
Hello from golang via ffi

Process finished with exit code 0

[chengxuncc]

I figured it out by decompiling c-shared library, the first parameter is a return value when Hello function doesn't have parameter but a return string:

// 1 return values, 0 parameter
GoString *__cdecl Hello(GoString *retstr)

There're decompiled functions with different numbers of parameters and return values:

// 0 return value, 2 parameters
void __fastcall Hello(GoString *p0, GoString *p1)

// 1 return value, 1 parameter
GoString *__fastcall Hello(GoString *retstr, GoString *p0, __int64 a3, GoString *a4)

// 1 return value, 2 parameters
GoString *__fastcall Hello(GoString *retstr, GoString *p0, GoString *p1, GoString *a4)

// 2 return values, 2 parameters
struct Hello_return {
    GoString r0;
    GoString r1;
};
Hello_return *__fastcall Hello(Hello_return *retstr, GoString *p0, GoString *p1, _QWORD *a4)

I did a test with 1 parameter and 1 return value situation:
go:

//export Hello
func Hello(p1 string) string {
    fmt.Println("called from dart ffi", p1)
    return "Hello from golang via ffi"
}

dart calls with 1 parameter:

typedef StringPtr = Pointer<GoString> Function(Pointer<GoString>);

void main() {
  final lib = DynamicLibrary.open('ffi.dll');
  final StringPtr Hello = lib.lookupFunction<StringPtr, StringPtr>("Hello");
  var p1 = GoString.fromString('p1');
  print(Hello(p1).ref.toString());
  print(p1.ref);
}

Output:

called from dart ffi p1
Hello from golang via ffi
Hello from golang via ffi

dart calls with 2 parameters:

typedef StringPtr = Pointer<GoString> Function(
    Pointer<GoString>, Pointer<GoString>);

void main() {
  final lib = DynamicLibrary.open('ffi.dll');
  final StringPtr Hello = lib.lookupFunction<StringPtr, StringPtr>("Hello");
  var p1 = GoString.fromString('p1');
  var p2 = GoString.fromString('p2');
  print(Hello(p1, p2).ref.toString());
  print(p1.ref);
  print(p2.ref);
}

Output:

called from dart ffi p2
Hello from golang via ffi
Hello from golang via ffi
p2

It seems work with passing 1 parameter on dart, but actually sharing memory with return value.
Normally calling Go function expecting returning GoString should pass a GoString parameter to receive return value.

[ghost]

Check out https://github.com/deromask/deromask. It work well with desktop and mobile. Basically, we passing data around via protobuf.

[chengxuncc]

I have given up trying Dart FFI:

1. Cross platform build issue.
2. Hard to access Flutter assets file handle from Dart.
3. Memory management is tricky.

[chengxuncc]

Thank you, I accepted @gedw99's advice. Using go mobile with grpc is a good idea for whom want to integrate Flutter and Golang on mobile.

• the golang process is a singleton and provides all data to the foreground flutter apps. So any data shared between two flutter apps stays in sync automatically. There is only one version of the truth on the device.

[james-lawrence]

grpc and gomobile is what I did as well in my app over a year ago.

only platform binding code I needed was to pass a user password and return a address:port so the flutter app could communicate to the golang service. ~200 lines of code for ios and android.

[gedw99]

@james-lawrence / @chengxuncc
Did you manage to get the gomobile code running as a background service on mobiles ?
I have not.
On Desktops it easy.