Sdk: webgl gl.readPixels works in dart2js but not in dartium vm

[DartBot]

_This issue was originally filed by [e-mail]_

---

What steps will reproduce the problem?

1. create a web application with the attached read_pixel.dart
2. run with dart2js -- click on the grey canvas -- it prints the correct pixel color: readPixels: x=14 y=384 color=[128, 128, 128, 255]
3. run in dartium -- click on the grey canvas -- it prints the WRONG pixel color: readPixels: x=14 y=384 color=[0, 0, 0, 0]

What is the expected output?
gl.readPixels should yield correct pixel color:
readPixels: x=14 y=384 color=[128, 128, 128, 255]

What do you see instead?
gl.readPixels yielding WRONG black pixel color:
readPixels: x=14 y=384 color=[0, 0, 0, 0]

What version of the product are you using?
Dart Editor version 0.5.20_r24275
Dart SDK version 0.5.20.4_r24275

On what operating system?
Windows 7 on 64 bit

---

Attachment:
read_pixel.dart (1.59 KB)

[anders-sandholm]

_Added Area-HTML label._

[blois]

_Set owner to [email protected]._
_Added this to the M6 milestone._

[iposva-google]

_Added Triaged label._

[efortuna]

is this already fixed?

[efortuna]

With the given test case it seems to be working now! I think we fixed this independently a little back.

---

_Set owner to @efortuna._
_Added CannotReproduce label._

[efortuna]

ps. Thanks for a great test/reproduction case.

[DartBot]

_This comment was originally written by [e-mail]_

---

Hmm, I am still hitting the issue: works in dart2js but not in dartium vm.

Did you get it to work in Dartium?

Just teste on:
64-bit Windows 7
Dart Editor version 0.6.21_r26639
Dart SDK version 0.6.21.3_r26639

[efortuna]

oops, I misread this bug. I thought the problem was in the dart2js version. Re-opening.

---

_Added Triaged label._

[larsbak]

_Removed this from the M6 milestone._
_Added this to the M7 milestone._

[efortuna]

update: this is a dartium bug, not dart:html. I've spent some time spelunking in the Chrome debugger, and haven't found the exact problem yet.

I suspect, however, than the issue lies in the bindings between Dart and native for readPixels -- the API is such that the result is in should be copied into the "color" array (the array called "color" in the attached test case), but I think that Dart doesn't realize this, and so it creates a new copy of the array to pass the parameters around in natively, and then never copies the calculated values back into the "color" array.

I'm willing to help out with debugging this or return to this issue, but in the meantime I'm going to pass it over to those with more Dartium expertise.

---

cc @vsmenon.
cc @jacob314.
_Removed the owner._
_Removed Area-HTML label._
_Added Area-Dartium label._

[DartBot]

_This comment was originally written by [e-mail]_

---

I am attaching another test case (pixel_framebuffer.dart) showing this issue affects only the default framebuffer (on-screen canvas). I mean, gl.readPixels in Dartium can correcly read pixel color from a custom off-screen render buffer.

---

Attachment:
pixel_framebuffer.dart (6.08 KB)

[vsmenon]

_Removed this from the M7 milestone._
_Added this to the M8 milestone._

[DartBot]

_This comment was originally written by [e-mail]_

---

I think the root cause of this issue lies here:

https://code.google.com/p/dart/issues/detail?id=9424

[iposva-google]

Vijay, can you please take a quick look whether there is something wrong with the bindings definitions?

---

_Set owner to @vsmenon._
_Removed Priority-Unassigned label._
_Added Priority-Medium, Accepted labels._

[vsmenon]

This is invoking dartToArrayBufferView, which has some known issues around releasing the underlying typed data pointer too early.

---

_Set owner to @a-siva._

[vsmenon]

_Removed this from the M8 milestone._
_Added this to the 1.1 milestone._

[a-siva]

After I fixed 9424 I tried this example and I now get
canvas 'webgl_canvas' created: width=600 height=400
WebGL: initialized
mouse offset: x=431 y=93
readPixels: x=431 y=307 color=[127, 127, 127, 255]

I don't quite get the values that you have indicated under expected, I am running the linux version of dartium though.

[DartBot]

_This comment was originally written by [e-mail]_

---

The test code is painting the canvas with gl.clearColor(0.5, 0.5, 0.5, 1.0);
0.5 maps to 127.5 in 0-255 space.
Thus I suppose my system is using a different rounding rule than yours.
Is it possible to download a bleeding edge SDK with your fix?

[a-siva]

The fix has been committed, any bleeding edge SDK post r30553 should have the fix.

---

_Added Fixed label._

[amaurel]

It seems that this bug is back !

What steps will reproduce the problem?

1. create a web application with the attached read_pixel.dart
2. run with dart2js -- click on the grey canvas -- it prints the correct pixel color: readPixels: x=14 y=384 color=[128, 128, 128, 255]
3. run in dartium -- click on the grey canvas -- it prints the WRONG pixel color: readPixels: x=14 y=384 color=[0, 0, 0, 0]

What is the expected output?
gl.readPixels should yield correct pixel color:
readPixels: x=14 y=384 color=[128, 128, 128, 255]

What do you see instead?
gl.readPixels yielding WRONG black pixel color:
readPixels: x=14 y=384 color=[0, 0, 0, 0]

What version of the product are you using?
Dart VM version: 1.13.1 (Thu Dec 17 14:42:19 2015) on "linux_x64"
Dartium Version 39.0.2171.0 (64-bit)

On what operating system?
Ubuntu 14.04

[jacob314]

I expect this is related to the switch to a JS interop based DOM for Dartium.
Do we have any tests for this issue?

[amaurel]

@jacob314
This issue can be tested using the code provided on first post by [email protected].
Tested on dartium 1.15.

[amaurel]

Is there any news on that issue ?
Shouldn't it be reopened ?

[muth02446]

This still seems to be broken - please re-open.
readPixels is an invaluable debugging aid.

[alan-knight]

The underlying problem is that the typed data doesn't get modified. I assume that what's happening internally is that a copy is being made and that's the one that gets the new data.

[alan-knight]

So yes, there's something going on with copying in the internals of Dartium. There isn't a quick fix, but you can work around it with a hack like the following, calling the JS version directly.

var _jsReadPixels;
get jsReadPixels {
  if (_jsReadPixels != null) return _jsReadPixels;
  js.context['eval'].apply(['function readPixelsDartium_(gl, x, y) { '
      '  var pixels = new Uint8Array(4); '
      '  gl.readPixels(x, y, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixels); '
      '  return pixels; '
      '}'
  ]);
  _jsReadPixels = js.context['readPixelsDartium_'];
  return _jsReadPixels;
}

readPixelsDartium(RenderingContext gl, x, y) {
  // If we're in dart2js, just call the method.
  if (1.0 is int) {
    Uint8List color = new Uint8List(4);
    gl.readPixels(
        x,
        y,
        1,
        1,
        RenderingContext.RGBA,
        RenderingContext.UNSIGNED_BYTE,
        color);
    return color;
  }
  var jsGl = new js.JsObject.fromBrowserObject(gl);
  return jsReadPixels.apply([jsGl, x, y]);
}

[rh-galaxy]

readPixels stopped working after Dartium 1.12.2.0 (this time around).

[jox81]

Any news about this problem ?

It should be re-opened...
Still got the problem...

Dart VM version: 1.20.1 (Wed Oct 12 15:07:45 2016) on "windows_x64"
Chromium Version 45.0.2454.104

[terrylucas]

Should work in Dartium 50 (uses JSInterop same as JS / dart2js).

[rh-galaxy]

Check the example in the first post, it doesn't work for me at least.
Chromium Version 50.0.2661.108

readPixels: x=282 y=311 color=[0, 0, 0, 0]

[rh-galaxy]

Do you have proof that it works for you @terrylucas?
Someone other than me that have tested the example in the first post?
I think this issue needs to be re-opened... yet again :/

[robertmuth]

I have a similar example using webgl2 and that does not work either.
The js version built with ddc is also broken.

[terrylucas]

@robertmuth webgl readPixels works in DDC and dart2js.

Webgl2 is a different issue.

[terrylucas]

Fixed the problem that's existed since Dartium 45 (ArrayBufferView not constructed when calling Javascript code) fixed in CL.

[terrylucas]

Fixed with commit 1265b4c6742487c025ebae97852d555cfd9d6522 (should be in 1.25 dev 2).

[terrylucas]

Fix in Dart 1.25.dev.6.0

[robertmuth]

This works beautifully now in WebGL2
Thanks Terry!

[JeromePuttemans]

Hello, I 've still got the problem in webgl1...
Does anyone else in the same case ?

Dart VM version: 1.25.0-dev.15.0 (Tue Aug 29 16:28:12 2017) on "windows_x64"
chromium Version 50.0.2661.108

in chromium : readPixels: x=151 y=130 color=[0, 0, 0, 0]
in chrome : readPixels: x=441 y=255 color=[128, 128, 128, 255]

Should this be re-opened ?

[robertmuth]

Hi Jerome,

just to clarify: when you say
in chrome : readPixels: x=441 y=255 color=[128, 128, 128, 255]
you are referring to the results of dart code compiled with dart2js and then run in chrome, right?

[JeromePuttemans]

Yes, Exactly