Godot: Screen capture not working (Android & iOS)

[Shin-NiL]

Godot version:
3.0-stable

OS/device including version:

Android 7.0, GPU: Mali-T860MP2

Issue description:
The screen capture official demo is not working on my device. After some debugging I've found that the img var is returning null at this point:

    get_viewport().set_clear_mode(Viewport.CLEAR_MODE_ONLY_NEXT_FRAME)
    # Let two frames pass to make sure the screen was captured
    yield(get_tree(), "idle_frame")
    yield(get_tree(), "idle_frame")

    # Retrieve the captured image
    var img = get_viewport().get_texture().get_data()

Steps to reproduce:
Just run the demo on a mobile device and press the "Capture screen" button.

Minimal reproduction project:
https://github.com/godotengine/godot-demo-projects/tree/master/viewport/screen_capture

[Shin-NiL]

As additional note, it's working fine on my PC (Windows 10).

[volzhs]

extends Sprite3D

func _ready():
    var view = $"Viewport"

    yield(get_tree(), "idle_frame")
    yield(get_tree(), "idle_frame")

    print("view.get_texture() = ",view.get_texture())
    print("view.get_texture().get_data() = ",view.get_texture().get_data())

02-26 06:15:30.276: I/godot(3112): view.get_texture() = [ViewportTexture:1815]
02-26 06:15:30.276: I/godot(3112): view.get_texture().get_data() = [Object:null]

it happens also with Viewport node which added in scene tree.
it works on x11 but not on Android device.

[Shin-NiL]

Not working on iPhone X simulator too. Same behavior,
var img = get_viewport().get_texture().get_data()
is returning null.

[pichinep]

I've tried everything, and the problem is to get_data on any texture, besides being the viewport one or not. (On pc/Linux works, on android it doesn't) I've also tried with the nightly build from https://hugo.pro/projects/godot-builds/, but no luck.

[juanpaexpedite]

Hi, how is this issue going on? this is a really important part for the game I'm finishing and I understand the issue is in saving the image, isn't it?

[ProbDenis]

I can confirm that Texture.get_data() doesn't work on Android. It always returns null.

[Shin-NiL]

As I can see get_data isn't working with ViewportTexture, it's working with ImageTexture though.

[ProbDenis]

Nope, for me it fails for ImageTextures too. Maybe it has something to do with the texture format or compression?

[Shin-NiL]

@ProbDenis it's strange, because it worked for me... I have no idea what is hapenning here :(

[pichinep]

I could solve this making a custom module for android and recomoiling templates... I dunno how yo share the code or push it in the git... any suggestions?

[Shin-NiL]

It's a core feature and should work on all platforms (as it always worked). The module solution can be used just as a workaround.

[peterjcarroll]

Adding a me too here. I'm calling .get_data on an ImageTexture. Works fine on windows but returns null on Android.

[henriquelalves]

Also having this same issue using Godot v3 (master); Shin-NiL Godot-Share module example works perfectly fine on PC (it uses Screen-Capture), but it won't work on Android as .get_data returns null.

There still no workaround for screen capture yet? This is somewhat crucial for mobile games sharing systems.

[Shin-NiL]

This is one of the reasons why I'm stuck in version 2.1.* :(

[reduz]

I will try to fix this for Godot 3.1

[jahd2602]

An option can be get back the Godot 2 function: Viewport.get_screen_capture() by copying the image that is being displayed in the screen here:
https://github.com/godotengine/godot/blob/d488c35efabee9d9317a91cdb905a460e7fce0af/servers/visual/visual_server_viewport.cpp#L320

[jabcross]

I've been trying to port code from 3.0 to 2.0, to try to get screenshot functionality into old android phones that don't support GLES3.

I've done this quick hack into viewport.cpp just to see if it works, but glReadPixels segfaults about one in six times.

I'm using the OP's MRP to test it.

I'd be grateful if anyone could give me some help.

Ref<Image> ViewportTexture::get_data() const {
    // 1 == GLES2
    if (OS::get_singleton()->get_current_video_driver() == 1){
        int width = this->get_width();
        int height = this->get_height();
        int size = width * height * 4;
        glPixelStorei(GL_PACK_ALIGNMENT, 4);
        Image *img = new Image(width, height, false , Image::FORMAT_RGBA8);
        img->lock();
        glReadPixels(0,0,width, height, GL_RGBA, GL_UNSIGNED_BYTE, img->write_lock.ptr());
        img->unlock();
        return Ref<Image>(img);
    }
    else{
        ERR_FAIL_COND_V(!vp, Ref<Image>());
        return VS::get_singleton()->texture_get_data(vp->texture_rid);
    }
}

[reduz]

The way to do this properly, IMO, when running on mobile, is to
1) Create a framebuffer and a texture, it could be an existing texture but I advise a new RGBA one, else many formats, like the compressed ones, can't be obtained.
2) Render the texture to it, and then reconvert the texture to the format. (if compressed, do not recompress)
3) Delete the framebuffer and newly created texture

@JFonS or @elasota Do you think you could lend a hand with this? I am not around until Tuesday.

[jabcross]

I've been following this SO question: https://stackoverflow.com/questions/1024603/opengl-es-render-to-texture#1024634

I noticed that most of this code is already implemented on render_target_create and render_target_set_size(). However, I'm stuck here. What function can I call that renders everything to the set render_target without resetting it?

[jabcross]

I feel like this should be working.

https://pastebin.com/HJ91ygEU

Any pointers?

[JFonS]

@jabcross I have quite some things to work on now, but I will eventually take a look at this issue.

Where exactly in the code are you doing these changes?

[jabcross]

@JFonS That's great to hear!

It's in the GLES part of rasterizer_storage_gles2::texture_get_data():

[chanon]

This is related so I'm commenting to this issue rather than opening a new one.

The issue is viewport.get_texture().get_data() can actually crash the game even in Windows.
It is somehow related to instancing a scene (not sure).

Anyways, here is reproduction code:
viewport_texture.zip

Just press the button.

Result (on master) is:

ERROR: CowData<class Ref<class Image> >::get: FATAL: Index p_index=0 out of size (size()=0)
   At: D:\godot\core/cowdata.h:146
CrashHandlerException: Program crashed
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[0] RasterizerStorageGLES3::texture_get_data (d:\godot\drivers\gles3\rasterizer_storage_gles3.cpp:1016)
[1] RasterizerStorageGLES3::texture_get_data (d:\godot\drivers\gles3\rasterizer_storage_gles3.cpp:1016)
[2] VisualServerRaster::texture_get_data (d:\godot\servers\visual\visual_server_raster.h:154)
[3] VisualServerWrapMT::texture_get_data (d:\godot\servers\visual\visual_server_wrap_mt.h:88)
[4] ViewportTexture::get_data (d:\godot\scene\main\viewport.cpp:129)
[5] MethodBind0RC<Texture,Ref<Image> >::call (d:\godot\core\method_bind.gen.inc:641)
[6] Object::call (d:\godot\core\object.cpp:968)
[7] Variant::call_ptr (d:\godot\core\variant_call.cpp:1049)
[8] GDScriptFunction::call (d:\godot\modules\gdscript\gdscript_function.cpp:1070)
[9] GDScriptInstance::call_multilevel (d:\godot\modules\gdscript\gdscript.cpp:1174)
[10] Node::_notification (d:\godot\scene\main\node.cpp:63)
[11] CanvasItem::_notificationv (d:\godot\scene\2d\canvas_item.h:140)
[12] Control::_notificationv (d:\godot\scene\gui\control.h:51)
[13] Object::notification (d:\godot\core\object.cpp:980)
[14] SceneTree::_notify_group_pause (d:\godot\scene\main\scene_tree.cpp:947)
[15] SceneTree::iteration (d:\godot\scene\main\scene_tree.cpp:474)
[16] Main::iteration (d:\godot\main\main.cpp:1818)
[17] OS_Windows::run (d:\godot\platform\windows\os_windows.cpp:2740)
[18] widechar_main (d:\godot\platform\windows\godot_win.cpp:150)
[19] _main (d:\godot\platform\windows\godot_win.cpp:174)
[20] main (d:\godot\platform\windows\godot_win.cpp:184)
[21] __scrt_common_main_seh (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283)
[22] BaseThreadInitThunk
-- END OF BACKTRACE --
ERROR: NetSocketPosix::_get_socket_error: Socket error: 10054
   At: drivers\unix\net_socket_posix.cpp:195

BTW I agree that this functionality is very important and a core functionality that should be usable.

[JFonS]

I implemented texture_get_data for OpenGL ES #23125 which should fix this issue. I need someone to test it on both Android and iOS to make sure it works properly.

[Shin-NiL]

Thanks @JFonS , I've tested your changes on my Android device using the official screen capture demo.

GLES 2
Worked just fine ;D

GLES 3
Almost there

* as a side note, the booting stage using GLES 2 is extremely slow (about 15s to open the app), but that's most likely another issue.

[jahd2602]

@JFonS We are working in an alternative implementation (nothing pull-requestable yet) and it looks very different than yours.

And here is how we are doing the screenshot: https://github.com/jahd2602/godot/commit/42fbfa7db13ba9ff675a20680ab7638c8067cc43

[JFonS]

Thanks for testing @Shin-NiL.

I'm not sure about the cause of the GLES3 mess. It looks like the shape is fine but the colors are interpreted in a wrong format. I will do some some changes later today to try and fix this.

@jahd2602 I'm by no means an OpenGL ES expert, I just implemented what reduz wrote:

The way to do this properly, IMO, when running on mobile, is to

1. Create a framebuffer and a texture, it could be an existing texture but I advise a new RGBA one, else many formats, like the compressed ones, can't be obtained.
2. Render the texture to it, and then reconvert the texture to the format. (if compressed, do not recompress)
3. Delete the framebuffer and newly created texture

If you solution works it will probably be better, since my PR is basically a hack.

[jabcross]

@JFonS our version (jahd2602, vnen and mine) works like it used to in Godot 2, which means it captures the whole framebuffer, and not a specific texture.

texture_get_data() should work for any texture, and not just viewports, right?

Thanks for your help, btw.

[JFonS]

@jabcross Yes, that's the idea. Although it still lacks support for mipmaps and compressed textures.

It looks like it works fine for screen caputre in GLES2 but I'm having some trouble getting it to work in GLES3.

[JFonS]

@Shin-NiL I updated my PR and it worked fine for me in GLES3. Have you tested the screen capture demo using a build from current master?

[Shin-NiL]

I'm sorry @JFonS it was my fault. I thought it was merged on alpha2. I've tested with master now and it's working fine with GLES2 and GLES3.

Thank you very much for your hard work ;)

[danielkotzer]

My version of Godot is 3.1.1 and it is not working on my iOS device, is it not fixed?