Ebiten: graphics: Shader API

[hajimehoshi]

EDIT: There is a proposal document for the shader language:

https://docs.google.com/document/d/1yVfqWY6B-yF0H3NOVff3r791_n2zUu8BUWdW9TuEorw/edit?usp=sharing

---

In Unity, there is a converter from HLSL to GLSL for portability

https://docs.unity3d.com/Manual/SL-ShadingLanguage.html

[hajimehoshi]

Actual example in other libs: https://pixijs.io/examples/#/basics/custom-filter.js

[hajimehoshi]

Another good example: https://twitter.com/stefanvlr/status/977574203913396224

[hajimehoshi]

As I need to implement Metal version first, I can't use GLSL. Let's postpone this task...

[hajimehoshi]

My current vague idea:

• Create a new intermediate language that can be compiled to GLSL, Metal shader, and other shaders.
• The language is a subset of Go, then can be parsed with go/ast
• Possibly, the code in the language can work as a regular Go program to demonstrate the shader result.

[hajimehoshi]

• All the embedded functions has names that start with small caps. This is similar to Go's embedded functions like make and it would be natural that they are generic functions (e.g., min can take both a float and a matrix)

[darkerbit]

There exists a converter from HLSL to GLSL, MojoShader, which is also used by Unity!

[aeperfilev]

Example of feature in the Pixel package: https://github.com/faiface/pixel/wiki/Using-a-custom-fragment-shader

[FryDay]

Has there been any progress made on this? It's the one feature that I feel is really missing from Ebiten.

[hajimehoshi]

No progress recently so far. Sorry but I don't have bandwidth. As Ebiten's graphics backends vary, shader is a quite difficult feature to define and implement.

[hajimehoshi]

OK I think I'll be working on shader this year, but I cannot guarantee.

[hajimehoshi]

https://jjagg.github.io/MonoGame-docfx/manual/content/custom_effects.html#custom-effects-1

Hmm, compiling a shader dynamically on MonoGame seems impossible :-/

[DianeLooney]

Do you consider this blocked by lack of MonoGame support? If not would a package that converts go into sl be helpful?

[hajimehoshi]

Do you consider this blocked by lack of MonoGame support?

No, I'd be able to create a special API to expose lower layers for MonoGame or such special environments, but I am not sure.

If not would a package that converts go into sl be helpful?

Yes and no, because Ebiten uses Metal instead of GL on macOS and iOS.

[DianeLooney]

Hm, I was under the impression that metal and opengl shader languages were pretty similar, but looking into it more that's not the case. Would a tool that attempts to target both be useful?

I'm going to be writing something similar to this shortly for gggv shortly to fix compatibility issues and would love to find a way to share my work. Being locked to MacOS's opengl versions on all platforms sucks.

[hajimehoshi]

Would a tool that attempts to target both be useful?

That sounds nice, but there are some items we need to consider for Ebiten:

• Ebiten's embedded shader has some functions e.g., retrieving a pixel in a specified region, or using color matrices. My idea is to enable to add a custom function to the existing embedded programs. Then, the tool would need to generate a program that can be embedded to other programs.
• Ebiten's OpenGL version is 2.1 and I don't plan to update it. The generator needs to generate compatible GLSL programs for such environments, and then some functions would not be available.
• My vague idea is to use go/ast for a shader language, but I am not sure this is a good idea.

I'm going to be writing something similar to this shortly for gggv shortly to fix compatibility issues and would love to find a way to share my work.

Nice. What is gggv btw?

Being locked to MacOS's opengl versions on all platforms sucks.

I plan to use DirectX on Windows later for some special environments (remote desktops and ARM windows), and might use Vulkan for Android for performance. This is not a matter of Apple anyway :-)

[DianeLooney]

Ebiten's embedded shader has some functions e.g., retrieving a pixel in a specified region, or using color matrices.

This sounds possible, I'll take a look at what's there and make sure to support that.

Ebiten's OpenGL version is 2.1 and I don't plan to update it.

Hm, I wasn't planning to target this old of a gl version but I could probably make it work.

Nice. What is gggv btw?

It's a livecode tool I've been working on. I develop it on linux and have broken so many features on windows and macos from inadvertently using a feature not supported by older opengl versions, or by writing shaders that compile only on linux, somehow.

Do you plan for this to be run at compile time or at runtime? I'm guessing runtime.

[hajimehoshi]

It's a livecode tool I've been working on. I develop it on linux and have broken so many features on windows and macos from inadvertently using a feature not supported by older opengl versions, or by writing shaders that compile only on linux, somehow.

Interesting! My guess is that whether a shader works or not depends on not only OSes but also a GPU and its driver.

Do you plan for this to be run at compile time or at runtime? I'm guessing runtime.

Yes, runtime.

[Zireael07]

Shader support is something that I feel is missing, but not that difficult considering https://github.com/go-gl/gl covers both GL and GLES.

[hajimehoshi]

Ebiten uses Metal on macOS and iOS. Ebiten might use DirectX on Windows in the future.

[Zireael07]

I found https://github.com/james4k/go-bgfx, which is a Go wrapper around https://github.com/bkaradzic/bgfx, which apparently supports WebGL and OpenGL and Metal (and some more)
Their shader docs: https://bkaradzic.github.io/bgfx/tools.html

[hajimehoshi]

Interesting, but this uses Cgo, which I want to avoid especially on Windows. I'll take a look anyway. Thank you!

[hajimehoshi]

https://love2d.org/wiki/love.graphics.newShader Lua LÖVE has its own shader language. Interesting.

[Zireael07]

Godot Engine also uses a custom variant of GLES: https://docs.godotengine.org/en/3.2/tutorials/shading/shading_reference/shading_language.html

[hajimehoshi]

https://gist.github.com/hajimehoshi/1e2453fe8bd0673ea35ceb9d9c5c4045

I've created a concept model of Go-like shader language based on the current Ebiten-internal shader. This of course will not be a Go program, but can be parsed. Another benefit is that go-fmt was available!

EDIT: This is just a concept model and no parser yet :-P

[hajimehoshi]

I'm writing a proposal document: https://docs.google.com/document/d/1yVfqWY6B-yF0H3NOVff3r791_n2zUu8BUWdW9TuEorw/edit?usp=sharing This is still draft.

[hajimehoshi]

Hm, this seems a tougher task than I expected first. Parsing the syntax was very easily done but analyzing the semantics means reimplementing a compiler's core part.

I'll continue this work for a while anyway.

[SolarLune]

Hello! Creating a brand new shader language is an interesting idea that appeals to my programmer core, haha. However, I wonder if it would be simpler and possible to adopt an intermediary language that can be transpiled into HLSL and GLSL - I saw that you mentioned SPIR-V in the proposal documentation, which seems like it was made for this purpose.

Would it be possible to simply convert, say, an existing shader file to SPIR-V, which then can be converted to GLSL, Metal and HLSL as necessary? Maybe just package the proper OS builds of SPIRV-Cross or glslang? There are pre-made builds of glslang on the repo there - maybe they could work for this purpose.

Actually making an additional language and transforming it into valid shader files for use on dedicated platforms seems like it would be a herculean task.

[hajimehoshi]

Hi, I understand your concern about a new language.

As far as I know, there are no pure-Go tools for SPIR-V. That's why I didn't adopt that.

Another reason is that there are some Ebiten-specific features. For the details, please take a look at the proposal document.

[hajimehoshi]

Another reason is that there are some Ebiten-specific features.

The biggest thing is that Ebiten uses internal automatic texture atlases, and this means that we cannot use regular shader programs in a straight way.

[hajimehoshi]

I think I'll take these steps for the shader:

1. Create a shader IR (intermediate representation) as AST
2. Create internal APIs to take an IR and execute the shader
3. Create a converter from Kage (Go-flavored shader) to the IR
4. Create a public API

I started with 3. first but I realized there was a big gap between Kage and its actual execution.

I think I'll close this issue after 2. is finished, and open new issues for each new task.

[hajimehoshi]

Implementing IR is almost done. See internal/shaderir package. I'll move on 2.

[hajimehoshi]

The item 2. is finished (only on OpenGL) :-) I'll break down this issue into small action items, then close this.

[hajimehoshi]

TestShader: image_test.go:309: opengl: shader compile failed: ERROR: 0:1: '' : No precision specified for (float)

Oops

[hajimehoshi]

The remaining items are:

• Metal implementation
• WebGL implementation
• Converting Kage to the IR
• Creating public APIs