Babylon.js: Some image textures appear rotated by default

[nbduke]

Repro playground: https://www.babylonjs-playground.com/#MXZNK2#2

In the above playground, I've placed two image textures with all default properties. The textures point to what is essentially the same image. The only different is that I opened the one on the right in Paint and saved it, whereas the one on the left is straight off a phone camera. As you can see, the left one appears rotated by default, whereas the right one has the correct orientation. However, if you download the left image or view it in a browser, it gets rendered the correct way.

This feels like a bug, especially since the left image renders properly outside the Playground.

[deltakosh]

woot! Really cool one..let me dig into that ;D

[deltakosh]

I'm gonna document the fix here because it is a WTF one :D
So here is what I see in Spector:

• texture1:
  

• texture2:
  

For some reasons, the loader rotate the first one even thought the texture size is CORRECT! (and we read it from the img file)

[deltakosh]

Firefox has no issue:

[deltakosh]

Ok I feel this is a Chromium issue

[nbduke]

@deltakosh Hmm, my repro Playground still has the issue in my Firefox (version 79.0).

[deltakosh]

https://bugs.chromium.org/p/chromium/issues/detail?id=1116060

[deltakosh]

Works for me on FF79:

[nbduke]

Well now I feel crazy because it's working for me.... Lol I need a coffee refill.

[deltakosh]

Please vote up the issue on chromium site

[deltakosh]

@nbduke apparently the image has orientation 6 in the EXIF which technically means this image is rotated 90 degs counter clockwise

So I wonder if this is really a bug or not

[nbduke]

@deltakosh But why does it render not rotated if you open the image in a browser tab? Or if you download it and open it in an image viewer on your computer.

[deltakosh]

because I guess the browser does not respect the EXIF info :)
But out of 4 browsers we have 2 who follows (chrome, opera) and 2 who don't (firefox, safari)

So I would recommend removing that info :)

[nbduke]

But my point is even Chrome appears to ignore the EXIF info if you just open the photo in its own tab (i.e. outside of the playground). Why is the behavior different if it's in WebGL vs plain HTML?

[nbduke]

Some further context on this issue. This originated with some pics one of our customers took on a phone. They expected them to render the same in Spaces as they did on their phone, but instead they showed up rotated. It feels weird to ask them to strip EXIF data (these aren't photographers and may not even know what that is).

[deltakosh]

Well now we know:https://bugs.chromium.org/p/chromium/issues/detail?id=1116060

And there is nothing we can do on our side unfortunately

[nbduke]

@deltakosh is there some property we can set on the texture to ignore EXIF? Or is that something you could add?

[deltakosh]

unfortunately no, we call texImage2D API and there is no control over image interpretation
(more general details: https://jdhao.github.io/2019/07/31/image_rotation_exif_info/)

I think the best option would be to process the image on the backend

[nbduke]

I guess what I don't understand is that if the browser is respecting the EXIF orientation, shouldn't it automatically rotate the raw image so that we see it the right way? That blog you linked me to certainly implies that:

And none of this explains why Chrome is inconsistent between rendering the image in HTML vs. WebGL.

[deltakosh]

feel free to comment on the chromium issue. My understanding is that only webgl is supporting it so far

[Popov72]

My understanding is that only webgl is supporting it so far

Or the other way around: webgl does not support it and the html rendering is ok.

As I understand it, the raw data are stored as a rotated image (as said in the link above) and a flag in the exif header does acknowledge that. And when it comes to render this picture on screen, the picture should be "unrotated" so that it appears the way the photograph took the picture: even if he rotated his phone to shoot a landscape (for eg), the landscape when displayed should appear normally and not rotated.

Taken from your pictures above with Spector:

Apart from being rotated, there's a problem with the picture: it misses the bottom of the original picture.

When "un"rotated:

Compared to the right picture:

You see that the picture is cut in its height for the first picture, which is abnormal, you should at least have all the data, even if rotated (after all, this picture is correctly displayed in HTML, so all the data are in the file). That makes me think there's a problem in WebGL and not in HTML, actually.

What do you think?

[deltakosh]

Well this is an excellent point. do you mind raising it in the chrome issue?