Sokol: Miscellaneous features required by a game engine

[bqqbarbhg]

Hi, thanks for the amazing libraries! I've needed to add some small features here and there for my game project. Most of these probably don't fit your vision of the libraries (and that's totally fine!) but I thought I'd list them anyway. The implementations of these are not the most robust (yet) but have been tested on Windows D3D11/GLCORE33, Windows/MacOS/Linux Chrome/Firefox WebGL2, macOS Metal, iOS Metal. I can clean any of these up for PR or you can just yoink some code yourself if you're interested!

I've kept some bqq_ namespacing in places to make merging with new versions of Sokol easier, just ignore those as they would be removed if integrated.

sokol_fetch.h: CURL for HTTP(S) requests on non-web platforms

https://github.com/bqqbarbhg/spear/blob/6e5017491fe3d9c4ac667a3d96585ccd55f7bc8a/src/ext/sokol/sokol_fetch_impl.h#L1673-L2001

Handle requests with http:// or https:// prefix using CURL. Initially I implemented support for it without requiring any build-time dependency on Windows by copy-pasting header bits and dynamically loading the functions with GetProcAddress(). This still exists in the code but has some issues, and should probably be removed if integrated. The SOKOL_CURL_STATIC define requires the user to include a couple of CURL headers before the implementation inclusion (I guess they could be included by sokol_fetch.h, at least optionally, if the feature is enabled).

sokol_fetch.h: NSURLSession for HTTP(S) requests on Apple platforms

https://github.com/bqqbarbhg/spear/blob/6e5017491fe3d9c4ac667a3d96585ccd55f7bc8a/src/ext/sokol/sokol_fetch_impl.h#L2003-L2037

I gave up on getting CURL to work on iOS so I implemented the internal "CURL API" using NSURLSession if __APPLE__ is defined.

sokol_app.h: Use high performance WebGL rendering context

https://github.com/bqqbarbhg/spear/blob/d59a2896afec32e22aadd1f544fada02f4a96e87/src/ext/sokol/sokol_app_impl.h#L2463-L2464

In my project I want to always request a high performance context so it's hard-coded to use powerPreference of EM_WEBGL_POWER_PREFERENCE_HIGH_PERFORMANCE. This could probably be a user-facing option in sapp_desc?

sokol_app.h: Target actual screen frame rate on iOS

https://github.com/bqqbarbhg/spear/blob/d59a2896afec32e22aadd1f544fada02f4a96e87/src/ext/sokol/sokol_app_impl.h#L1606

Use UIScreen.mainScreen.maximumFramesPerSecond instead of fixed 60Hz to get 120Hz on iPad Pro. Again a hard-coded default but could be an option?

sokol_gfx.h: Additional sampled texture formats

Additional texture formats, mostly just sRGB versions of existing ones. ASTC would have a lot more size options, but listing them all would add a ton of new formats.

| Format | D3D11 | GLCORE33 | GLES2 | GLES3 | METAL_MACOS | METAL_IOS |
| --- | --- | --- | --- | --- | --- | --- |
SG_PIXELFORMAT_BQQ_SRGBA8 | Yes | Yes | No* | Yes | Yes | Yes
SG_PIXELFORMAT_BQQ_BC1_SRGB | Yes | Ext | Ext | Ext | Yes | No
SG_PIXELFORMAT_BQQ_BC3_SRGB | Yes | Ext | Ext | Ext | Yes | No
SG_PIXELFORMAT_BQQ_BC7_SRGB | Yes | Ext | Ext | Ext | Yes | No
SG_PIXELFORMAT_BQQ_ASTC_4X4_RGBA | No | Ext | Ext | Ext | No | Yes
SG_PIXELFORMAT_BQQ_ASTC_4X4_SRGB | No | Ext | Ext | Ext | No | Yes
SG_PIXELFORMAT_BQQ_ASTC_8X8_RGBA | No | Ext | Ext | Ext | No | Yes
SG_PIXELFORMAT_BQQ_ASTC_8X8_SRGB | No | Ext | Ext | Ext | No | Yes

* Could be supported as Ext using EXT_sRGB which is pretty widely supported.

sokol_gfx.h: Resource / render pass labels

Use ID3D11*_SetPrivateData() (D3D11) / .label = (Metal) to name images and buffers. Use ID3DUserDefinedAnnotation_Begin/EndEvent() (D3D11) / .label = (Metal) to tag render pass names. Could add support for GL debug names too, but I don't think WebGL supports them (well it doesn't have a debugger so it wouldn't really be useful).

sokol_gfx.h: Update subimage of an existing image

https://github.com/bqqbarbhg/spear/blob/d59a2896afec32e22aadd1f544fada02f4a96e87/src/ext/sokol/sokol_gfx_impl.h#L10130-L10137

If an image is created with bqq_copy_target set to true in desc you can later update regions of it using sg_bqq_update_subimage(). The implementation is not the most optimal, but in practice using glTexSubImage2D() / UpdateSubresource / replaceRegion seems not to stall the GPU.

sokol_gfx.h: Copy image regions

https://github.com/bqqbarbhg/spear/blob/d59a2896afec32e22aadd1f544fada02f4a96e87/src/ext/sokol/sokol_gfx.h#L1612-L1632

This is a wild one, it allows you to copy regions of an image to another one on the GPU. Like the above extension it requires bqq_copy_target to be set for the destination image. The reason for the batch API is that setting up the copying requires some heavyweight setup on OpenGL and some on Metal. The function is a bit limited on GLES2 where you can only copy to the top-level mip of an image.

[floooh]

Wow these are great changes! I'll just give some very quick feedback for now, but I think all of them should be considered for inclusion one way or another (either through PRs or I simply add them when I get around to it)

sokol_fetch.h: CURL for HTTP(S) requests on non-web platforms
sokol_fetch.h: NSURLSession for HTTP(S) requests on Apple platforms

Excellent!

sokol_app.h: Use high performance WebGL rendering context

Also a great fix, but I'd make it configurable via sapp_desc (similar to the other sapp_desc.html5_* items that already exist)

sokol_app.h: Target actual screen frame rate on iOS

This is a wonderful fix, I wasn't aware of the maximumFramesPerSecond UIScreen member yet!

PS: we probably need something better than the current sapp_desc.swap_interval though, because a game that should run with 60fps on a 120Hz display shouldn't fall back to 30fps on a 60Hz display... there's also a separate related discussion about being able to disable vsync.

sokol_gfx.h: Additional sampled texture formats

I was a bit hesitant to add the SRGB pixel formats because I had various problems in GL with them in the past, but I guess it makes sense to add them (my advice so far was to handle the SRGB conversion in the pixel shader as needed).

ASTC: no objections of course :)

Thanks for doing the cross-platform research!

sokol_gfx.h: Resource / render pass labels

I was considering this as a low-priority task. We need to make sure that the string pointer passed into D3D11 and Metal don't have to remain valid for the lifetime of the resources (otherwise we need to copy them into small buffers in the sokol-gfx resource structs, like I did in sokol_gfx_imgui.h)

sokol_gfx.h: Update subimage of an existing image
sokol_gfx.h: Copy image regions

At some time in the hopefully not-to-distant-future I want to overhaul the resource-updating functions in sokol-gfx completely, as this is currently the weakest part of the entire API IMHO. I've collected a few general ideas in my head while working on the Metal and WebGPU backends, but I need to get some order into that first... quick brain dump:

• Writing dynamic "per-frame-data" (e.g. CPU-computed vertex- or pixel-data which is recomputed each frame) would be separate from copying 'persistent' content into buffer and image resources, this should primarily be a mapping-API where you get a pointer back to write into (and maybe a separate "write-function" for convenience, but which involves a separate memcpy). On most APIs (all except WebGL/GLES2) this (the map-function) can prevent a memory copy, which currently always happens. The data written with this map/write API will never persist, and may not even end up in sokol-gfx resource objects (at least for buffers).

• A copy-API for copying 'persistent' CPU data into buffer and image resources, also used for partial updating.

• A separate copy-API (maybe find yet another name) for copying data between resources, at least buffer-to-buffer and image-to-image.

• Ideally also a group of functions to copy data from GPU resources back to the CPU side... but this will probably come without any performance guarantees (e.g. they might block).

(when I write "API", I mean a group of functions, not a separate sokol header)

[bqqbarbhg]

PS: we probably need something better than the current sapp_desc.swap_interval though, because a game that should run with 60fps on a 120Hz display shouldn't fall back to 30fps on a 60Hz display... there's also a separate related discussion about being able to disable vsync.

This is actually pretty tricky from an API standpoint, for my use case something like maximum_frames_per_second would be the most useful. Could calculate the swap interval internally from the monitor refresh frequency and that setting.

I was a bit hesitant to add the SRGB pixel formats because I had various problems in GL with them in the past

In my experience at least WebGL2 sRGB support is solid for sampling as a texture, though there might be mipmap weirdness I have dodged (https://twitter.com/won3d/status/1287047957544685569). Unfortunately I had to drop WebGL1 support so I haven't been able to test EXT_sRGB on WebGL1 yet.

On the other hand rendering to sRGB framebuffers is super wonky on WebGL1/2 as you can't choose between linear/sRGB like you can on desktop using GL_FRAMEBUFFER_SRGB. The only thing you can do is query GL_FRAMEBUFFER_ATTACHMENT_COLOR_ENCODING_EXT and hope that it supports sRGB. In my current implementation sRGB formats are reported to support only sampling on GL.

We need to make sure that the string pointer passed into D3D11 and Metal don't have to remain valid for the lifetime of the resources

My current implementation actually does support this, and I use dynamic resource names a lot! Only render passes need to keep the name cached in Sokol and I have (arbitrarily chosen) 64 character inline strings in the _sg_pass_t struct for that. D3D11 requires labels to be UTF-16 so for render passes I cache the UTF-16 encoded result.

https://github.com/bqqbarbhg/spear/blob/d59a2896afec32e22aadd1f544fada02f4a96e87/src/ext/sokol/sokol_gfx_impl.h#L691

At some time in the hopefully not-to-distant-future I want to overhaul the resource-updating functions in sokol-gfx completely, as this is currently the weakest part of the entire API IMHO. I've collected a few general ideas in my head while working on the Metal and WebGPU backends, but I need to get some order into that first... quick brain dump:

The resource update overhaul sounds super useful! I'm not certain but it might be that WebGL1/GLES2 sadly doesn't support buffer copies. With buffer copies you could do partial buffer updates with staging resources in a neat way:

void *data;

// This returns a buffer handle that doesn't need to be freed and is only valid for this frame
// Backends could linearly sub-allocate these out of larger buffers if possible.
// `data` is either mapped or temporarily allocated if it's not supported.
sg_buffer_t temp_buffer = sg_push_mapped_temp_buffer(&data, num_bytes); 

write_data(data);

// This is either a no-op, unmap, or glBufferSubData() and free()
sg_commit_mapped_buffer(temp_buffer, data);

// sg_copy_buffer(dst_buffer, dst_offset, src_buffer, src_offset, num_bytes)
// As `temp_buffer` is uploaded to the GPU as well there's no risk for a stall here
// even if we're currently using `persistent_buffer` for rendering.
sg_copy_buffer(persistent_buffer, offset, temp_buffer, 0, num_bytes);

You could also do something similar and still support WebGL1 like this:

// WebGL1: No GPU resource, temporary CPU allocation
// WebGL2:  Temporary GPU buffer (slice), temporary CPU allocation
// Other APIs: Temporary GPU buffer, mapped CPU pointer
void *data = sg_begin_buffer_update(num_bytes);

write_data(data);

// WebGL1: glBufferSubData(persistent) and hope the driver doesn't cause a stall
// WebGL2: glBufferSubData(temporary) glCopyBufferSubData(persistent, temporary)
// Other APIs: CopyBuffer(persistent, temporary)
sg_commit_buffer_update(persistent_buffer, offset, data);

[bqqbarbhg]

One more half-baked feature for consideration: I needed to reset the "key down" state when the user deactivates the window. Currently implemented using WM_SETFOCUS/WM_KILLFOCUS on Win32 and window "focus" and "blur" events on Emscripten. I'd guess other platforms would be doable as well but I don't have time to investigate deeper at the moment.

https://github.com/bqqbarbhg/spear/commit/f09bde5f0df199cb8ec9482b7bea423755519bff?ts=4