Three.js: Ocean example breaks in WebVR mode

[spacorum]

Description of the problem

Using Water.js (from the ocean shader example), there is a problem when entering WebVR mode (tested with Firefox + Oculus Rift and also with the Chrome + WebVR emulation). The mirror shader doesn´t rotate in sync with the rest of the scene, and it´s strechted. Also, if you rotate the camera enough, the sphere on one side will be completely covered by the ocean shader, while the other sphere will be completely visible (like there was a unique ocean shared between boths cameras, I can´t explain better).

It looks like the whole reflection shader only reflects the original camera, but not the VR ones (notice the reflection in the middle, is the sphere you would see with VR disabled).

The problem seems to be in the _scope.onBeforeRender()_ function in Water.js (commenting this code fixes the ocean size/position/rotation, but obviously there is no mirror shader on it (only a dark plane without any reflections).

• [jsfiddle] (https://jsfiddle.net/spacorum/61oqycz7/)
• [jsfiddle] (https://fiddle.jshell.net/spacorum/61oqycz7/show/)
  fullscreen direct link to allow WebVR emulation controls on Chrome

Three.js version

• [ ] Dev
• [X] r107

Browser

• [ ] All of them
• [x] Chrome
• [x] Firefox
• [ ] Internet Explorer

OS

• [ ] All of them
• [x] Windows
• [ ] macOS
• [ ] Linux
• [ ] Android
• [ ] iOS

Hardware Requirements (graphics card, VR Device, ...)

Graphics card: NVIDIA GeForce GTX 1070
VR Device: Oculus Rift on Firefox (it can be replicated using Chrome + WebVR emulation)

[Mugen87]

Can you please also verify the sandbox example with your system? Does the mirror also break there?

https://threejs.org/examples/webvr_sandbox

[spacorum]

Yes, the mirror in the sandbox example also breaks in both cases (Firefox +
Oculus, Chrome + WebVR emulator). The mirror displays stretched and badly
located/rotated. As the ocean, it works well in standard mode, but not with WebVR enabled.

[Mugen87]

I can reproduce with Chrome 76 Beta/WebXR and a Pixel 1. The rendering of the mirror in webvr_sandbox is broken. Especially the location of the mirror in 3D space is totally wrong.

Everything is fine in non-XR mode.

[Mugen87]

Maybe related to #16367?

/cc @fernandojsg

[spacorum]

Thanks, @Mugen87 this fixes the problem in the webvr_sandbox mirror.
But how could this be applied to the Water.js example? It doesn´t use the Reflector/Refractor files, so the problem is still there. Should this new code be used in scope.onBeforeRender() somehow?

[Mugen87]

Ops sorry, I forgot that the ocean example uses THREE.Water and not THREE.Water2 (which internally uses THREE.Reflector).

Should this new code be used in scope.onBeforeRender() somehow?

I could imaging that the same code from the PR fixes the issue in THREE.Water. Can you place it at the very end of onBeforeRender() and see if that solves the problem? Otherwise I can verify this tomorrow by myself.

[Mugen87]

In general we have not verified that all example code can be used in XR. So depending on what example you convert to a XR demo, you might encounter more files that need XR-related patches.

Hence, I'm going to change this issue to a feature request since THREE.Water never supported XR so far.

[Mugen87]

Um, I've tested your fiddles on a Pixel 1 but it's not possible to successfully enter VR. The site hangs up. It seems the mentioned enhancement to onBeforeRender() is not sufficient to run the example in VR.

[spacorum]

Can you take a look on a computer instead of the mobile?
The fix magically worked for me on Water.js (tested on desktop and laptop), so maybe it´s a problem with Pixel? The reflection shows properly located/rotated after adding your code at the very end of scope.onBeforeRender():

// Restore viewport
var viewport = camera.viewport;
if ( viewport !== undefined ) {
        renderer.state.viewport( viewport );
}

Now I can see the ocean perfectly on both cameras (left and right).
It goes smooth on Oculus Rift and also with Chrome XR emulator.

[Mugen87]

Interesting...yeah, the problem could be related to Daydream.

Do you want to make a PR with the enhancement of Water.js?

[spacorum]

Sure. Never made one tbh, but I´m on it.

[Mugen87]

Great! I'll try to make another test with Oculus Quest the next days...