Libgdx: Grayscale png not properly loaded

[realitix]

Issue details

I want to use a grayscale image (with palette) like this one: image | Format detail.

When I load this image, Pixmap.Format is set to Alpha, good! But in my shader, when I sample the alpha channel, it's just white. I tried with LuminanceAlpha and Intensity on all channels and it's not working either.

When loading image with Texture(FileHandle, Format, Mipmap), if I force format to be RGBA8888, it works in the alpha channel of my shader! So I have to force the RGBA8888 with a grayscale image (wasting memory).

Version of LibGDX and/or relevant dependencies

1.9.3-Snapshot
OpenglEs3 enabled

Please select the affected platforms

• [ ] Android
• [ ] iOS
• [ ] HTML/GWT
• [X] Windows
• [X] Linux
• [ ] MacOS

[realitix]

More information on this bug:

In GLTexture, I updated uploadImageData with:

Gdx.gl.glTexImage2D(target, miplevel, GL30.GL_R8, pixmap.getWidth(), pixmap.getHeight(), 0, GL30.GL_RED, GL30.GL_UNSIGNED_BYTE, pixmap.getPixels());

If I sample the red channel in my shader, it works!
So I think the problem come from the ALPHA type not supported with gles3.

ALPHA format is deprecated for OpenGL here but with OpenGLES 3, it should work here.

On desktop, when I enable gl30, do I use OpenGLES3 or OpenGL>3.3 ?

[realitix]

I tested it on GL20 and it works with GL_ALPHA.

So here is the problem :

On Desktop, GL_ALPHA, GL_INTENSITY are removed when useGL30 = true.
We have to use GL_RED to be able to fetch the grayscale texture.

Currently, GDX2DPixmap format only supports GL_ALPHA and GL_INTENSITY, so you can't use grayscale image with gl30 (workaround: force RGBA8888).

Solution 1 : Add support for GL_RED in Gdx2DPixmap based on gl version. (very heavy to implement, jni, gwt emulation...)
Solution 2 : In GLTexture, when uploading texture, convert GL_ALPHA with GL_RED if gl30. (very light but dirty)

[realitix]

@xoppa @Tom-Ski @MobiDevelop What do you think ?

[realitix]

SSCCE

Copy this image into asset folder.

package com.badlogic.gdx.tests;

import com.badlogic.gdx.Gdx;
import com.badlogic.gdx.graphics.GL20;
import com.badlogic.gdx.graphics.Mesh;
import com.badlogic.gdx.graphics.Pixmap;
import com.badlogic.gdx.graphics.Pixmap.Format;
import com.badlogic.gdx.graphics.Texture;
import com.badlogic.gdx.graphics.VertexAttribute;
import com.badlogic.gdx.graphics.VertexAttributes.Usage;
import com.badlogic.gdx.graphics.glutils.ShaderProgram;
import com.badlogic.gdx.tests.utils.GdxTest;

public class GrayscaleTest extends GdxTest {
    ShaderProgram shader;
    Texture texture;
    Mesh mesh;

    @Override
    public void create () {
        String vertexShader = "attribute vec4 a_position;   \n" + "attribute vec2 a_texCoord;   \n"
            + "varying vec2 v_texCoord;     \n" + "void main()                  \n" + "{                            \n"
            + "   gl_Position = a_position; \n" + "   v_texCoord = a_texCoord;  \n" + "}                            \n";

        String fragmentShader = "#ifdef GL_ES\n" + "precision mediump float;\n" + "#endif\n"
            + "varying vec2 v_texCoord;\n" + "uniform sampler2D s_texture;\n"
            + "void main()\n"
            + "{\n"
            + "  gl_FragColor = vec4(vec3(texture2D( s_texture, v_texCoord ).a), 1.0);\n"
            + "}\n";
        shader = new ShaderProgram(vertexShader, fragmentShader);
        mesh = new Mesh(true, 4, 6, new VertexAttribute(Usage.Position, 2, "a_position"), new VertexAttribute(
            Usage.TextureCoordinates, 2, "a_texCoord"));
        float[] vertices = {-0.5f, 0.5f, // Position 0
            0.0f, 0.0f, // TexCoord 0
            -0.5f, -0.5f, // Position 1
            0.0f, 1.0f, // TexCoord 1
            0.5f, -0.5f, // Position 2
            1.0f, 1.0f, // TexCoord 2
            0.5f, 0.5f, // Position 3
            1.0f, 0.0f // TexCoord 3
        };
        short[] indices = {0, 1, 2, 0, 2, 3};
        mesh.setVertices(vertices);
        mesh.setIndices(indices);

        Pixmap pixmap = new Pixmap(Gdx.files.internal("test_grayscale.png"));
        System.out.println("Pixmap format :"+pixmap.getFormat());
        texture = new Texture(pixmap);
        pixmap.dispose();
    }

    public void render () {
        Gdx.gl20.glViewport(0, 0, Gdx.graphics.getBackBufferWidth(), Gdx.graphics.getBackBufferHeight());
        Gdx.gl20.glClear(GL20.GL_COLOR_BUFFER_BIT);

        Gdx.gl20.glActiveTexture(GL20.GL_TEXTURE0);
        texture.bind();

        shader.begin();
        shader.setUniformi("s_texture", 0);

        mesh.render(shader, GL20.GL_TRIANGLES);

        shader.end();
    }
}

On Desktop:

If GL30 enabled => White texture

If GL30 disabled => Grayscale texture

[xoppa]

Thanks. Can reproduce on desktop. On desktop OpenGL is always used, the version is configurable in the config. Afaik OpenGL ES 3.0 should be compatible (a subset of) OpenGL 4.3 (not GLSL ES btw). But I guess you found a case where it isn't. I don't have a gles3 android device to test on, so cant verify it indeed works on gles3, but the docs suggest it would. Perhaps someone can verify that?

I don't think there's a good solution for this, as in: we cannot abstract this incompatibility away because it requires modifying the shader. GLES3 adds GL_RED though, so I guess the fix would be to add GL_RED support when gles3 is used (the html backend doesnt support gles3 yet). It would be up to the user to modify the shader accordingly then.

[realitix]

@xoppa Take a look at that: http://docs.gl/es3/glTexParameter

On gles3 (not on gles2), we can use the GL_TEXTURE_SWIZZLE_* property.

What do you think about that ? Where is the best place to use this ?
Anyway, the default shader is not compatible with gles3-glsl

[pixelherodev]

Does the average libGDX user know how to modify the shader?

On 04/30/2016 06:01 PM, Xoppa wrote:

Thanks. Can reproduce on desktop. On desktop OpenGL is always used,
the version is configurable in the config. Afaik OpenGL ES 3.0 should
be compatible (a subset of) OpenGL 4.3 (not GLSL ES btw). But I guess
you found a case where it isn't. I don't have a gles3 android device
to test on, so cant verify it indeed works on gles3, but the docs
suggest it would. Perhaps someone can verify that?

I don't think there's a good solution for this, as in: we cannot
abstract this incompatibility away because it requires modifying the
shader. GLES3 adds |GL_RED| though, so I guess the fix would be to add
|GL_RED| support when gles3 is used (the html backend doesnt support
gles3 yet). It would be up to the user to modify the shader
accordingly then.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/libgdx/libgdx/issues/4029#issuecomment-215997883

[realitix]

Hi np511. If a user don't know how to modify a shader, he shouldn't use gles 3 for now.

[xoppa]

Yeah swizzle could work. Would have to look into the detail of the implementation, but rather keep it as much at one place as possible. Afaik the default shader works fine on gles3 though.

[JFixby]

If a user don't know how to modify a shader, he shouldn't use gles 3 for now.

Reason?

[Darkyenus]

@JFixby GLES3 brings only advanced features over GLES2. Someone who is not able to modify a shader won't be able to use these features anyway.

[sp-1234]

Someone who is not able to modify a shader has no business developing games, tbh.

[BenMcLean]

Someone who is not able to modify a shader has no business developing games, tbh.

Come now, that is not fair. GLSL is a very odd and finicky language and requires understanding some concepts that someone who makes simple 2D games may have never encountered before. On top of this, WebGL only supports a subset of it's features.

I myself have been getting help making modifying and tweaking shaders for the first time over the past few months. I'd say one definitely doesn't have to already know how to modify a shader to start developing games.

[sp-1234]

GLSL is a very odd and finicky language

Not much weirder than Java, and certainly much less weirder than C. It doesn't even have any ways to shoot yourself in the foot.
Before GLSL, one had to write in GPU assembly and this is indeed weird. But this is not needed anymore.

These concepts (how GPU programmable pipeline works) are not really hard, it's entirely possible to grok basics in one week if the person already has proper experience in programming in general, I think… and once that's done, GLSL is pretty much walk in the park, it allows to directly express most things that are needed in shaders and if something is not right, compiler gives reasonable error messages.

[tommyettinger]

So, as far as I can tell, this is a valid issue still, but there's nothing we can do about it... Optimizing for GPU memory usage is not easy. As far as I can tell, there isn't a generally-portable way to load a single-channel image into GPU memory on all platforms (HTML doesn't support GL_RED).

But if you want grayscale images for file size reasons:

If you're using PNG output color mode 0 (grayscale) or 4 (grayscale with alpha), it seems like PNG compressors can't do as much for file-size with those, and color mode 3 (palette, or indexed color) generally makes a smaller image as long as there is at most one transparent color. So the only reason to prefer 8bpp grayscale over 8bpp palette is if the image is so tiny that the palette takes up a significant part of the file, and since we had better be using texture atlases, that should never happen in a real game. Shaders won't ever interact with a PNG directly; they only see what has been loaded onto the GPU, and if the PNG was palette-based with 256 shades of gray, the GPU will have the same contents as if it had loaded a grayscale PNG, if both using the same format.

I am able to load this 200-something-color paletted atlas using RGBA8888 format specified in the atlas, and at least that works on HTML and desktop. I might be able to use RGB888. This does use more memory on the GPU than a single-channel format, but the portability is important for me.

I typically use PNGOUT to losslessly compress atlases. It's kinda slow, but a lot faster than zopfliPNG, and according to its readme, zopfliPNG only gets 0.5% smaller file sizes typically.

Back to the issue at hand, memory usage, I think xoppa answered this well 4 years ago:

I don't think there's a good solution for this, as in: we cannot abstract this incompatibility away because it requires modifying the shader. GLES3 adds GL_RED though, so I guess the fix would be to add GL_RED support when gles3 is used (the html backend doesnt support gles3 yet). It would be up to the user to modify the shader accordingly then.

There hasn't been any PR or similar activity trying to implement the complicated-looking swizzle code, but if someone was willing to write one without breaking compatibility, I would absolutely merge that PR. I'll leave this issue open for now; it isn't completely resolved without someone maybe partially trying to implement swizzling on GLES 3.0 and reporting back whether it's feasible.

[NathanSweet]

I typically use PNGOUT to losslessly compress atlases.

@tommyettinger FWIW, I haven't tried PNGOUT but oxipng works well and is fast.