Godot: ShaderGLES2 Errors

[avencherus]

Godot version:
2a10e30

OS/device including version:

Win10 64-bit

OpenGL ES 2.0 Renderer: GeForce 9600 GT/PCIe/SSE2
OpenGL ES 2.0 Batching: ON

Issue description:

Was tabbed back into work in the editor and saw these errors in the console that I haven't seen before, and not sure what caused them.

EDIT - Log

OpenGL ES 2.0 Renderer: GeForce 9600 GT/PCIe/SSE2
OpenGL ES 2.0 Batching: ON
    OPTIONS
    max_join_item_commands 16
    colored_vertex_format_threshold 0.25
    batch_buffer_size 16384
    light_scissor_area_threshold 1
    debug_flash False
    diagnose_frame False

**ERROR**: Condition "!cc" is true. Returned: 0
   At: drivers\gles2\shader_gles2.cpp:214:ShaderGLES2::get_current_version() - Condition "!cc" is true. Returned: 0
**ERROR**: Condition "!version" is true. Returned: false
   At: drivers\gles2\shader_gles2.cpp:88:ShaderGLES2::bind() - Condition "!version" is true. Returned: false
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1

Steps to reproduce:

Currently not known. Will post back if I can isolate it better.

Minimal reproduction project:
N/A

[lawnjelly]

Will investigate when we get a MRP, and looking through to see what could cause this.

@clayjohn could this possibly be leaving conditional CanvasShaderGLES2::USE_TEXTURE_RECT set (or vice versa)? I'm not very familiar with how the shader versioning works, I'll look into it.

If this is the batching, and one of the conditionals, it should be easy to fix once we have a repeatable project.

Just to clarify, do you think it was the IDE causing this (the OS console) or the project being run (the Godot IDE console + maybe the OS console)?

[avencherus]

It's proving elusive to catch. I'm not sure about the console distinctions, so I don't think I can give you a good answer. My knowledge on Godot runs thin here.

It was running outside of an IDE at that time. I think it was being thrown by the tools/editor version of Godot, as the app wasn't running.

Any ideas on what I might probe at to try to reproduce it?

[lawnjelly]

Ah it does sound like it was occurring just with the editor rather than with a running project. Those bugs are far more tricky to repeat and diagnose! :grin:

Did you have a particular project loaded at the time? Even running the editor it will be displaying the preview tabs of the project and that can help pin down what might be causing it. Any hints you might have as to what you had on screen (was it a 2d project, 3d etc) might be a clue.

If you can't repeat it don't worry too much, it is very helpful that you have pinpointed the lines where an error were occurring and I can do some tests and see if I can recreate the error artificially. I'll also familiarise myself with the shader versioning.

At this stage we can't say for sure it was due to the batching (it could have been a rare bug that occurs anyway) but I'll assume it is the batching for now until I can see otherwise.

[avencherus]

Yeah, just throwing it out there, in case it is relevant or helpful for what you're working on.

It was a 2D project in GLES2. I opened it, tabbed away to look at something else for a minute, when I returned it noticed those in the console.

I'll take another look and see if I can coax it out.

[avencherus]

Ghost problems. 👻

Tried many things. I also went back to the log, and there were also some later warnings about items that had gone missing from an instanced scene. (It cleaned itself up.)

So I tried to replicate that to see if it maybe influenced something during startup, but nothing I'm trying has reproduced it yet.

A momentary lapse in the OS providing the version information at startup?

What would happen if it failed to get the version in a release build?

    if (!version->ok) { //broken, unable to bind (do not throw error, you saw it before already when it failed compilation).
        glUseProgram(0);
        return false;
    }

[lawnjelly]

Ghost problems.
Oh I'm all too familiar with these 'heisenbugs'! :grinning:

So I tried to replicate that to see if it maybe influenced something during startup, but nothing I'm trying has reproduced it yet.

It is okay, don't worry too much, sometimes these things are hard to pin down until someone later finds a repeatable version. It could be anything, alt-tabbing, a race condition with background import of files etc. I'll try and have a systematic look through once I have tracked down another bug I'm working on, I'm not familiar enough with the shader versioning to guess what might be happening.

What would happen if it failed to get the version in a release build?

Nothing too scary, I suspect it would just not draw the items using that shader (and perhaps only on a single frame, blink and you miss it!).

[lawnjelly]

Just to quote from clayjohn on this, he knows much more than me about this:

the shader bug just means a version failed to compile. We need an MRP to do anything. The issue could be with the users shader code, and they unknowingly fixed it and only ever saw one error.

[avencherus]

Thanks for the follow up. 😸 The project doesn't yet have any custom shaders. Only one or two codeless godot canvas material type shaders (made via inspector), where I set the blend mode to add for a glowing effect on some sprites.

Wondering if I should close this. It doesn't seem like it would be much of an issue, and nothing to work on without pinpointing the shader involved.

Perhaps a bit off topic. Are there any ways to retroactively debug these kinds of errors right after they happen? I'm not too experienced with C++ debugging and profiling, but it seems like I would have to have a breakpoint set in advance, and some of the lines with error macros I can't conditionally breakpoint in VS, because they rely the direct return values of functions. I would like to avoid the mess and time loss of writing in hacky lines and recompiling to do the typical debugging.

[clayjohn]

The best way to debug an error like this is to reproduce it. If you can figure out what happened in the project that led to this error appearing then we can figure out what caused it.

[lawnjelly]

Perhaps a bit off topic. Are there any ways to retroactively debug these kinds of errors right after they happen? I'm not too experienced with C++ debugging and profiling, but it seems like I would have to have a breakpoint set in advance, and some of the lines with error macros I can't conditionally breakpoint in VS, because they rely the direct return values of functions. I would like to avoid the mess and time loss of writing in hacky lines and recompiling to do the typical debugging.

Well Godot tends to use its own error macros which fail gracefully. This is good in the sense that most errors won't stop a game / the IDE from running (for end users), however for debugging in development at the time it is usually better to have something like an assert which halts the program and allows you to debug with a stack trace, so you can find out what caused a condition immediately. This is especially important in 'heisenbugs', that rarely occur, or are unpredictable, which can be quite common in rendering.

There actually is an argument for having a #define or setting that allows a _development_ build to replace things like ERR_FAIL with asserts that will halt immediately, allowing us to catch such bugs quicker. Perhaps there is such a macro already. I'll try and ask on IRC later.

[avencherus]

@lawnjelly What is this argument that forces assertion?

[Calinou]

@avencherus I think @lawnjelly said we should add a SCons flag to abort on errors. There doesn't appear to be one currently.

[bojidar-bg]

Note that it is possible to set a GDB breakpoint on error_macros.cpp:82 or so -- that way it would stop right after printing each error, allowing one to use the bt and frame commands to examine the state of the program.

Running through rr might be a simpler way to go back to heisenbugs, however.

[avencherus]

@bojidar-bg Thanks for this info, I haven't used GDB, but will have it bookmarked for later. And I was wondering a good place to break for error macros.

I will close this issue, as I haven't seen it since. I'll re-open if there is a way to reproduce it or it happens again and I catch it in a debugger.