Cesium: WebGL 2 and WebGL Extensions Roadmap

[pjcozzi]

Extensions in Canary

• [x] [WEBGL_debug_shaders](https://www.khronos.org/registry/webgl/extensions/WEBGL_debug_shaders/) and WEBGL_debug_renderer_info - no longer require privileged access.
• [ ] [OES_element_index_uint](http://www.khronos.org/registry/webgl/extensions/OES_element_index_uint) - already used by createVertexArrayFromGeometry, but could also be used by billboards and polylines that use vertex buffer directly.
• [x] [OES_texture_half_float](http://www.khronos.org/registry/webgl/extensions/OES_texture_half_float/)
• [x] [OES_texture_float_linear](http://www.khronos.org/registry/webgl/extensions/OES_texture_float_linear/) and OES_texture_half_float_linear for completeness.
• [x] [WEBGL_compressed_texture_s3tc](http://www.khronos.org/registry/webgl/extensions/WEBGL_compressed_texture_s3tc/) - for glTF.
• [x] [EXT_frag_depth](http://www.khronos.org/registry/webgl/extensions/EXT_frag_depth/) (still draft) - opaque ray-traced geometry, e.g., ellipsoids, sensors, etc.

Draft WebGL Extensions

• [ ] [WEBGL_color_buffer_float](http://www.khronos.org/registry/webgl/extensions/WEBGL_color_buffer_float/) and EXT_color_buffer_half_float for HDR (I think).
• [ ] [WEBGL_compressed_texture_atc](http://www.khronos.org/registry/webgl/extensions/WEBGL_compressed_texture_atc/) and WEBGL_compressed_texture_pvrtc - for glTF
• [ ] [WEBGL_draw_buffers](http://www.khronos.org/registry/webgl/extensions/WEBGL_draw_buffers/) - first for combing color and pick passes in a single pass. Later, with the post-processing framework, we can combine other passes like glow and velocity. Shadow and reflection passes will still be separate since they are rendered with different view parameters.
• [ ] [WEBGL_shared_resources](http://www.khronos.org/registry/webgl/extensions/WEBGL_shared_resources/) - for multiple canvases, but not web workers?

Proposed WebGL Extensions

• [ ] [ANGLE_timer_query](http://www.khronos.org/registry/webgl/extensions/proposals/ANGLE_timer_query/), for profiling tools.
• [ ] [WEBGL_compressed_texture_etc1](http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_compressed_texture_etc1/) - for glTF.
• [ ] [WEBGL_dynamic_texture](http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/) - for video textures.

OpenGL ES 3.0 Features

• [ ] Multisampled Renderbuffers - AA without using FXAA.
• [ ] Half-float vertex attributes, i.e., GL_OES_vertex_half_float.
• [ ] Uniform buffer objects - for backing uniforms with buffers based on update frequency. Also see #649.
• [ ] Sampler objects - the Cesium abstractions already look like this to some extent.
• [ ] Immutable textures - reduce driver validation (EXT_texture_storage).
• [x] Instancing, i.e., ANGLE_instanced_arrays, for batching geometry and models.
• [ ] Primitive restart - batch render line strips instead of individual lines.
• [ ] Transform feedback - perhaps for particle systems.
• [ ] Program binaries - to reduce shader compile times. (Not in WebGL 2.0)

More on ES 3.0

• OpenGL ES 3.0 at SIGGRAPH 2012
• OpenGL ES 3.0 – Halti has arrived

More on WebGL 2.0

• What's coming in WebGL 2.0
• WebGL 2 Specification
• Mozilla Platform/GFX/WebGL2

WebGL 2 Demos

• http://toji.github.io/webgl2-particles/

Done

• [x] [OES_vertex_array_object](http://www.khronos.org/registry/webgl/extensions/OES_vertex_array_object/) - #766.

[pjcozzi]

Uniform buffer objects

The renderer could always expose UBOs and emulate them with traditional uniforms or uniform arrays if they are not supported. This will require some GLSL macros or shader patching for the uniform/uniform-buffer declarations.

Automatic uniforms can be grouped into uniform buffers based on update frequency: per frame and per object (model-matrix).

Each command can provide two uniform buffer: one for uniforms that don't change often and one for uniforms that do. This maps to technique (don't change) and material (do change) parameters. Or one uniform buffer might be fine since the command creator won't have to figure out how to organize them.

[pjcozzi]

Cesium Implementation Status (after #3094)

• OES_standard_derivatives is disabled. dFdx, dFdy and fwidth are not recognized when compiling the shader.
  
  
  
  • There should be one test failure for materials because the PolylineArrowMaterial uses standard derivatives.
  
  
• EXT_frag_depth is disabled. gl_FragDepth is not recognized when compiling the shader (neither is gl_FragDepthEXT).
• OES_texture_float is disabled. Could not create a texture with PixelDatatype.FLOAT.
• WEBGL_depth_texture is disabled. Could not attach a depth-stencil texture to a FBO.

[pjcozzi]

• WebGL 2.0 talk from SIGGRAPH Asia 2015 - https://docs.google.com/presentation/d/1Orx0GB0cQcYhHkYsaEcoo5js3c5-pv7ahPniIRIzzfg/edit?usp=sharing

[pjcozzi]

• It will be easy enough to do by hand, but here's a tool to convert WebGL 1 to WebGL 2 shaders: https://github.com/Jam3/glsl-100-to-300/

[pjcozzi]

WebGL 2 Samples

• https://github.com/WebGLSamples/WebGL2Samples
• https://floooh.github.io/oryol-webgl2/

[pjcozzi]

• http://webgl2fundamentals.org/

[pjcozzi]

WebGL 2 articles on the RTR blog:

• WebGL 2 Basics
• WebGL 2: New Features

[pjcozzi]

Texture Array: we could use TEXTURE_2D_ARRAY in some cases where texture atlases are used. It will be simpler and great for dynamic updates when the size doesn't change, #2319

[pjcozzi]

• MDN has doc on WebGL 2 - see the Firefox 51 release

[pjcozzi]

• WebGL 2 lands in Firefox

[pjcozzi]

• WebGL 2 from WebGL 1
• 4.1 Backwards Incompatibility

[pjcozzi]

• WebGL 2 Algorithm Examples

[pjcozzi]

• WebGL 2 Webinar Slides

[pjcozzi]

Hardware PCF for soft shadows

[lilleyse]

• Mipmapping restrictions gone. No longer need to upscale in Model.js
• Point cloud should use integer attributes for certain things
• 3D Tiles batch table can avoid uploading a pick texture. https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Cesium3DTileBatchTable.js#L1371-L1374. This is also doable now but won't work for pick ids that exceed 16777217 (last int floating point can represent... would this ever happen?).

[bagnell]

For FXAA, use textureLod and textureLodOffset like the original shader. Consider using textureSize instead of a uniform.

[pjcozzi]

• A minimal, WebGL 2-only rendering library https://tsherif.github.io/picogl.js/

[pjcozzi]

• Use native uint32 to avoid the pick texture for 3D Tiles, see createPickTexture in Cesium3DTileBatchTable.js and createPickIds in ModelInstanceCollection.js

[pjcozzi]

• WebGL 2.0 Masterclass from Web3D 2017

[pjcozzi]

Likely run our existing GLSL through modernizeShader, #5623, to generate GLSL ES 3.0, and then have a bit of a DSL (domain-specific language) with marcos/etc to handle differences like varying vs in/out. I'm sure it will not be that simple and some thing will have to be GLSL ES 3.0 specific, but we do not want to limit ourselves to only using GLSL ES 1.0 over the long haul, and I believe that engines like Unity do exactly this.

[pjcozzi]

• Reaching the Largest Gaming Platform of All: The Web. WebGL, WebVR and glTF. GDC 2017

[pjcozzi]

• NanoGL - thin WebGL 2 wrapper

[lilleyse]

If running a webgl2 context bypass texture upscaling code in Model when generating mipmaps.

EDIT: forgot this was already added above.

[lilleyse]

See https://github.com/AnalyticalGraphicsInc/cesium/pull/6314 which enabled WebGL 2 by default but hasn't been tested enough yet and had gone stale.

[lilleyse]

Possible terrain loading bug in WebGL 2: https://github.com/AnalyticalGraphicsInc/cesium/issues/6756

[OmarShehata]

For anyone trying to use CesiumJS with WebGL 2, note this is not currently officially supported, but is on the roadmap. I'm consolidating open issues with running CesiumJS with WebGL 2 here.

For reference, you can activate WebGL 2 by initializing the viewer as follows:

var viewer = new Cesium.Viewer('cesiumContainer', {
    contextOptions: {
        requestWebgl2: true
    }
})

Known issues with WebGL 2

• [x] Classification example https://github.com/CesiumGS/cesium/issues/8629
• [x] Post processing https://github.com/CesiumGS/cesium/issues/7952
• [x] Incorrect depth testing https://github.com/CesiumGS/cesium/issues/7942
• [x] Error when using clipping planes https://github.com/CesiumGS/cesium/issues/7712
• [x] HDR https://github.com/CesiumGS/cesium/issues/7647
• [x] Zooming https://github.com/CesiumGS/cesium/issues/797#issuecomment-645660881
• [ ] Ambient occlusion artifacts https://github.com/CesiumGS/cesium/issues/797#issuecomment-646916121
• [x] Cesium Inspector slowness https://github.com/CesiumGS/cesium/issues/797#issuecomment-646919297

[mramato]

The original issue description is very out of date, if we're going to use this to track WebGL2 implementation, someone should update it with updated thoughts and plan of action.

[lilleyse]

@mramato 👍

That'll be on me an @YoussefV once we start the WebGL 2 / MSAA work.

[lilleyse]

In WebGL 2 the camera zooms towards the globe instead of towards the geometry that the mouse is hovering over. This seems like a bug in either ScreenSpaceCameraController or scene.pickPosition.

Sandcastle

WebGL 1

WebGL 2

[dennisadams]

@lilleyse
When ScreenSpaceCameraController needs a pickPosition, it calls both scene.pickPositionWorldCoordinates and globe.pickWorldCoordinates and chooses the closer of the two, assuming both are defined.
In our case, scene.pickPositionWorldCoordinates returns undefined and so ScreenSpaceCameraController uses the globe pick instead.
I played around a little and the problem seems to be that the depth texture is empty.
With the help of this error, I tried hard-coding a NEAREST filtering and it worked alright (even the error was gone). So I guess it's some problem with linear minification/magnification filtering.
I tried it with the changes from 8969, so that can also be a factor.

[lilleyse]

@dennisadams awesome, that fixes it for me as well. Just to confirm you added nearest filtering to the depth-stencil texture?

  globeDepth._depthStencilTexture = new Texture({
    context: context,
    width: width,
    height: height,
    pixelFormat: PixelFormat.DEPTH_STENCIL,
    pixelDatatype: PixelDatatype.UNSIGNED_INT_24_8,
    sampler: Sampler.NEAREST // Fixes the zoom
  });

[baothientran]

@dennisadams @lilleyse That also fixes the Silhouette post processing as well. Also I look it up, and in WebGL 2, it seems depth texture cannot have linear filtering. That may be a potential problem with other depth textures used by Cesium as well

[dennisadams]

Just to confirm you added nearest filtering to the depth-stencil texture?

I went on a safe bet - hard-coded the Texture constructor.

[lilleyse]

Labels look a bit different in WebGL 2. Click the images to see the difference.

WebGL 2

WebGL 1

Sandcastle

[lilleyse]

The AO post process has artifacts: Local sandcastle.

[lilleyse]

Polyline arrows start to get squashed. Local sandcastle

[lilleyse]

Cesium Inspector is extremely show when "Show Frustums" is checked. Local sandcastle

[lilleyse]

Contour lines are much thinner in WebGL 2. Local sandcastle.

First thing I would look into is GL_OES_standard_derivatives which is another built-in define that's part of WebGL 1 but maybe not WebGL 2 (I haven't checked). GL_EXT_frag_depth is another one we should investigate.

[lilleyse]

Alright that's all I've uncovered so far by going through each sandcastle example with WebGL2 enabled. @baothientran I recomend starting with https://github.com/CesiumGS/cesium/issues/797#issuecomment-646920630 and seeing what other bugs are fixed in the process.

[lilleyse]

@baothientran since the number of bugs fixes was starting to grow too much I consolidated them into a single CHANGES.md entry: https://github.com/CesiumGS/cesium/commit/65757f8b7dce74d1ffbf9d61ea10d6e4508b86f2

[lilleyse]

Also I updated the checklist above: https://github.com/CesiumGS/cesium/issues/797#issuecomment-617189407

[lilleyse]

• [ ] modernizeShader is too slow. This is something we should address before defaulting to WebGL 2.
  
  
  
  • [ ] This may mean upgrading all shaders to GLSL ES 3.0 with some DSL (aka @pjcozzi's comment https://github.com/CesiumGS/cesium/issues/797#issuecomment-316990165)
  
  
  • [ ] Or see https://github.com/CesiumGS/cesium/issues/9201 for a workaround