Three.js: Rendering with equiangular environment map?

[richardmonette]

@mrdoob A few of the offline renderers that I am familiar with take HDR light probes in equiangular/latlng format. Quick question: is this supported in 3 right now? If not, I'll take a look at making an example that demonstrates this. cc @bhouston

[bhouston]

We support HDR light probes either as FP32, FP16 or a packed format, but in CubeMap format or CubeUV2 format (a flat format.) I would not suggest adding support for latlong, but instead add conversion to CubeMap. The reason is that we already convolve properly in CubeMap and CubeUV2 but with latlong we would have to come up with a new convolution method in order to model roughness.

[bhouston]

Convolving latlong is not simple though because its projection is highly distortive at the polar regions.

[bhouston]

Demo of light probe in *.hdr cubemap format, convolved and packed and then used for PBR lighting:

https://threejs.org/examples/#webgl_materials_envmaps_hdr

What you want is this same pipeline but start with a latlong image in EXR or HDR format. Should be relatively straightforward to do....

[WestLangley]

@bhouston Do we support HDR cubemap backgrounds? I don't think that has been implemented.

Oh, and your proposal would be great! :)

[richardmonette]

Working out the last bugs now but wanted to quickly tease that I've almost got this working!

I started from @bhouston 's HDR environment mapping example, and have add an EXR equiangular -> HDR cubemap generator, which feeds into the existing PMREM generator. You may recognize the light probe is https://github.com/mrdoob/three.js/blob/dev/examples/textures/piz_compressed.exr which is from my prior EXR work. All the conversion is handled right in 3, which obviates the need for any 3rd party tools or pre-processing.

Hoping to have a PR up in the coming week!

[bhouston]

Very nice! If you could ensure that the API you allow you to switch input image type between various formats like HDR, EXR, LDR PNG in latlong, that would amazing and probably the proper API.

[bhouston]

How slow is the conversion between equiangular and a cubemap? Are you doing it in CPU or on the GPU? I know it is possible to do on the GPU relatively fast. Basically one writes a shader and execute it on every pixel of a cubemap and you convert back to the equiangular map to read the pixels. Generally that should work I think and be nearly instantaneously fast, although it involve a little manual interpolation if you were using an encoding like RGBM or RGBE.

[richardmonette]

If you could ensure that the API you allow you to switch input image type between various formats like HDR, EXR, LDR PNG in latlong, that would amazing and probably the proper API.

👍

Are you doing it in CPU or on the GPU?

I'm doing it on the GPU. I'll get a more accurate timing, but it feels nearly instant on start-up.

[richardmonette]

Did some timing of the equiangular EXR -> cube map conversion, it's ~28ms