Go-flutter: Add support for external OpenGL textures

[pchampio]

The flutter engine ABI supports external OpenGL textures allowing the embedder to draw to OpenGL directly. engine#7087

It's needed to implement the VideoPlayer.

[ghost]

Will also be useful for doing 3D graphics and webrtc.

[cloudwebrtc]

@Drakirus @gedw99
I have done a texture patch on flutter/engine based on GLFW, it seems to work very well, but I don't know how to port to go-flutter, here is my Texture rendering code, maybe it will help you.
https://github.com/cloudwebrtc/flutter-engine/blob/texture_for_glfw/shell/platform/glfw/external_texture_gl.cc

[pchampio]

@cloudwebrtc Is the EventChannel require for the Texture to work?

[cloudwebrtc]

@Drakirus
I think EventChannel is needed, because when the video frame changes in size and aspect ratio, the flutter side needs to adjust the display scale or fit, EventChannel is used to send notifications to the flutter side.
In addition, if we want to adapt WebRTC.PeerConnection, or other plugins with many event sources, EventChannel is needed.

[pchampio]

Hi @cloudwebrtc I'm seeing progress on flutter/engine about the textureAPI :+1:.
I'm trying to implement the textureAPI for go-flutter based your CPP code.

While implementing the API, I hit a wall.

My understanding was that you register a texture using: FlutterEngineRegisterExternalTexture, then mark a new frame with FlutterDesktopMarkExternalTextureFrameAvailable. At some point, in between or after those calls, the flutter-engine calls you back using the gl_external_texture_frame_callback. My issue is that gl_external_texture_frame_callback NEVER gets called.
Am my missing something?
Here is my progress on the implementation: commit

[cloudwebrtc]

@Drakirus Sorry to reply so late, I see you in the code RegisterExternalTexture(textureId: 400), and after a while MarkExternalTextureFrameAvailable(textureId: 600), this place textureId should be a fixed value of int64_t when registering once, for example: textureId = 0, when your The plugin generates a new video frame, only need to execute MarkExternalTextureFrameAvailable(texture: 0), then the flutter engine will call GLExternalTextureFrameCallback(texture: 0, ...).

frameI := int64(0)
for !a.window.ShouldClose() {
    frameI++
    glfw.WaitEventsTimeout(0.016) // timeout to get 60fps-ish iterations            
    if frameI == 400 {
        print("REGISTER: ")
        print(a.engine.RegisterExternalTexture(frameI))
    }
    if frameI == 600 {
        print("New Frame: ")
        print(a.engine.MarkExternalTextureFrameAvailable(frameI))
    }

change to:

frameI := int64(0)
textureId := int64(0)
print(a.engine.RegisterExternalTexture(textureId))
for !a.window.ShouldClose() {
    frameI++
    glfw.WaitEventsTimeout(0.016) // timeout to get 60fps-ish iterations            
    print("New Frame: ")
    print(a.engine.MarkExternalTextureFrameAvailable(textureId))

[pchampio]

@cloudwebrtc Thanks for your reply!
The testing code that I pushed was wrong, I've updated it: commit.

I'm still facing the same issue: gl_external_texture_frame_callback never gets called. :disappointed:

I've updated the code to use a bare C handler code, OnAcquireExternalTexture never get called.

The terminal output looks great:

[....]
Texture: register result: true
go-flutter: no handler found for channel flutter/isolate
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
Texture: New Frame result: true
go-flutter: closing application

[cloudwebrtc]

@Drakirus Your changes are correct, just add Texture(textureId: 1) to flutter lib/main.dart.

            child: Container(
              color: Colors.lightBlueAccent,
              child: Column(
                mainAxisAlignment: MainAxisAlignment.center,
                children: <Widget>[
                  Texture(textureId: 1),
                  Text('You have pointed at this box this many times:'),
                  Text(
                    '$_enterCounter Entries\n$_exitCounter Exits',
                    style: Theme.of(context).textTheme.display1,
                  ),
                  Text(
                    'The cursor is here: (${x.toStringAsFixed(2)}, ${y.toStringAsFixed(2)})',
                  ),
                ],
              ),
            ),

I tested it under linux, OnAcquireExternalTexture is called.

[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
Texture: New Frame result: true
=== OnAcquireExternalTexture: 1
Unknown pixel config 0x0
[ERROR:flutter/shell/platform/embedder/embedder.cc(578)] Could not create external texture.
go-flutter: closing application

[pchampio]

Texture(textureId: 1) that what I was missing, Thanks!!

[pchampio]

Making progress:

A demo is available at: https://github.com/go-flutter-desktop/examples/tree/feature/gl_texture/texture_tutorial, use hover run -b '@beta' to try it out.

@cloudwebrtc everything was implemented thanks to you, big thanks!

[cloudwebrtc]

Wow, that's awesome.

[pchampio]

Updated the demo. Added an very simple videoplayer

[pchampio]

fix taged in tag: v0.29.0