Sokol: Scene rendering

[amerkoleci]

Hi,

I was thinking of using sokol gfx for rendering in my engine (https://github.com/amerkoleci/alimer), as per today I have own D3D11, D3D12 and vulkan backend, I was thinking about how it integrates in real scene rendering with material etc and maybe shader permutation?

How to compute the pipeline from different mesh, material

[floooh]

Just one suggestion how it could work:

• a basic material would be shader + pipeline + texture slots (e.g. diffuse, specular, normal texture), for permutations (e.g. the same material as alpha-blended, or vertex-skinned) you would need additional shaders and pipelines. The actual material permutation would also define the uniform buffer layout.

• a mesh would be a vertex-buffer, index buffer and 'primitive groups' (aka submesh ranges) - the range of primitives that can be assigned one material, defined by the base_element and num_elements args of sg_draw()

• a "renderable" would have a material index, a mesh index, a submesh-index into the mesh, and the actual textures that go into the material's texture slot, the renderable is basically where materials are applied to geometry

• a multi-material object would be a collection of several renderables

...etc etc...

...once I'm done with the sokol-shdc shader compiler tool I'm planning to write a GLTF-sample, this would basically contain a material-system as defined by the GLTF format.

[amerkoleci]

I see, what I see "problematic" is the pipeline has the vertex layout information which is part of mesh, in that case probably you need to combine/permute the pipeline with material/shader + mesh part.

[floooh]

Hmm, in a way, the mesh doesn't have any vertex format baked in, it's just a bag of data, the actual format is whatever the pipeline object says it is, the mesh just has to have all the required data somewhere available.

Example:

Let's say there are a couple of material variations:

• with lighting or not (lighting would require normals, binormals, ...)
• with skinning or not (requires per-vertex bone-indices and skin-weights)
• ...

The mesh data would need to provide the "sum" of the required vertex components, e.g. when all vertex components are interleaved one vertex in the mesh would look like this:

struct vertex_all_t {
  vec3 position;
  vec2 uv;
  vec3 normal;
  vec3 binormal;
  vec4 bone_indices;
  vec4 bone_weights;
};

The pipeline for "non-skinned, unlit" only requires the vertex positions and texture coords in the vertex layout:

     .layout = {
            .buffers[0].stride = sizeof(vertex_all_t)
            .attrs = {
                [0] = { .name="position", .offset=offsetof(vertex_all_t, position) .format=..., },
                [1] = { .name="uv", .offset=offsetof(vertex_all_t, uv), .format=... }
            }
        },

The pipeline for "non-skinned, lit" would additionally require the normals and binormals:

     .layout = {
            .buffers[0].stride = sizeof(vertex_all_t)
            .attrs = {
                [0] = { .name="position", .offset=offsetof(vertex_all_t, position) .format=..., },
                [1] = { .name="uv", .offset=offsetof(vertex_all_t, uv), .format=... },
                [2] = { .name="normal", .offset=offsetof(vertex_all_t, normal), .format=... },
                [3] = { .name="binormal", ... }
            }
        },

...and the same idea for the other 2 pipelines ("skinned + unlit" and "skinned + lit"), those would additionally require the bone-indices and -weights.

You can also keep the vertex components in separate chunks inside the vertex buffer (e.g. position and uvs interleaved in their own chunk, followed by normal and binormal interleaved in their own chunk, followed by a 3rd chunk with interleaved bone_indices and bone_weights), this avoids the gaps between per-vertex-data and is apparently more efficient for the GPU fetching the vertex data. In this case you need to use the different buffer slots in the pipeline's vertex layout, and the verte_buffer_offset-member in sg_bindings, so it's a bit more complicated to setup.

[floooh]

...closing this. I've started to add a FAQ label to question and will mark them "closed" to unclutter the open-tickets list a bit.