Webxr: Add support for Ambient Light Events

[ranbuch]

Chrome doesn't support Ambient Light Events.

Also, while rendering immersive-ar a webGL canvas I'm unable to get a screenshot of the rendered scene (from obvious reasons).

There is no way to adjust the scene to the real world.

The best thing that a web-xr developer can ask is to be able to get a screenshot and dynamically create an environment map.

That way we will be able to reflect items from the real world on 3D assets.

An array of dominant colors that are being captured by the camera might be helpful to determine what environment map we should use (out of our existing maps).

Ambient Light Events can help us adjust the scene exposure.

Thanks.

[klausw]

Have you seen the lighting estimation proposal?
https://immersive-web.github.io/lighting-estimation/ . Chrome supports an
earlier version of this proposal as an experiment, but I'm not up to date
on details. @toji would know more.

On Thu, Aug 6, 2020, 19:05 Ran Buchnik [e-mail] wrote:

Chrome doesn't support Ambient Light Events
https://developer.mozilla.org/en-US/docs/Web/API/DeviceLightEvent/Using_light_sensors
.

Also, while rendering immersive-ar a webGL canvas I'm unable to get a
screenshot of the rendered scene (from obvious reasons).

There is no way to adjust the scene to the real world.

The best thing that a web-xr developer can ask is to be able to get a
screenshot and dynamically create an environment map.

That way we will be able to reflect items from the real world on 3D assets.

An array of dominant colors that are being captured by the camera might be
helpful to determine what environment map we should use (out of our
existing maps).

Ambient Light Events can help us adjust the scene exposure.

Thanks.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/immersive-web/webxr/issues/1109, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKT5X45LOUCMFCULQXLMUTR7NONHANCNFSM4PXFC4ZQ
.

[Manishearth]

Yeah, this is a matter for the lighting estimation module, which is being developed.

[ranbuch]

Looks promising. Would I be able to convert the values from this API to three-js Light Probe or something of a sort?

[toji]

Would I be able to convert the values from this API to three-js Light Probe or something of a sort?

That's exactly the intent, yes. The data isn't specific to Three, of course, but Three is well positioned to consume it. It looks something like this:

let threeLightProbe = new THREE.LightProbe();

// Every frame:
let lightEstimate = xrFrame.getLightEstimate(webxrLightProbe);
threeLightProbe.sh.fromArray(lightEstimate.sphericalHarmonicsCoefficients);

[ranbuch]

I'm trying to utilize this feature:
xrOptions.optionalFeatures = ['light-estimation'];
But getting a console warning:
Unrecognized feature requested: light-estimation
I'm using Chrome Canary version 88.0.4306.0 on Android 11 (Samsung Galaxy S20 Ultra).

Have you already implemented this feature?

Is there any estimation for when it will be ready?

Thanks.

[klausw]

@ranbuch , lighting estimation is still an experimental feature, see https://github.com/immersive-web/lighting-estimation .

You can enable it in Chrome Canary by enabling "WebXR Incubations" in chrome://flags, and track the feature status at https://www.chromestatus.com/feature/5704707957850112 . Please be aware that the API is still likely to change since it's under active development.

[ranbuch]

Thanks @klausw

I have also registered to the origin trial.

I see that WebXR Depth Sensing is also avaleble on origin trial, clearly you have been busy 💪

[ranbuch]

@toji While supporting lighting estimation will create a better experience it still not sufficient to cover for the lack of AmbientLightSensor support, at least when using three.js LightProbe implementation.

It does add lights according to the colors of the texture pixels but it won't occlude light.

If, for example, the user will place the model on a dark room the model still appears lit.

[toji]

If, for example, the user will place the model on a dark room the model still appears lit.

The values that lighting estimation exposes absolutely do account for this scenario. All three light sources exposed by the API (directional, ambient spherical harmonics, environment cube map) modulate their brightness based on the amount of light in the user's environment. It may even make use of the hardware's ambient light sensor, I don't know for sure!

If you have other virtual lights in your scene then it obviously won't modify those (which may be what you meant by occluding?)

[ranbuch]

Thanks @toji

If you have other virtual lights in your scene then it obviously won't modify those (which may be what you meant by occluding?)

Yes, that's what I meant.

All three light sources exposed by the API (directional, ambient spherical harmonics, environment cube map) modulate their brightness based on the amount of light in the user's environment

If I understand you currectly if I will use LightProbe it will change the amount of light that the other light sources emits.

Meaning, in case a cirtun fragment is mapped to a bright pixel on the light probe he will be brighter.

If however a cirtun fragment is mapped to a dimmed pixel on the light probe will he be dimmer?

I'm pretty sure he won't be.

And if I'm correct there's no way to dim the scene in case we are in a dark room.

The model will look like he is emitting light and will just look out of place.