React-three-fiber: Getting screen coords via `vector.project(camera)` doesn't work on first call of `useUpdate`

[granmoe]

See codesandbox.

I need to get the canvas coordinates for the top-left and bottom-right vertices of a plane and then store them in redux (this is to facilitate grabbing an image of the portion of the canvas covered by that plane). I thought using the recommended three.js approach but within a useUpdate would work, but it appears to bomb for some reason on initial render. I've noticed that it does work after a resize.

A snippet of the codesandbox:

  const {
    camera,
    size: { width: canvasWidth, height: canvasHeight }
  } = useThree()

  const ref = useUpdate(
    geometry => {
      const vector = geometry.vertices[0].clone()
      vector.project(camera)
      console.log(vector) // { x: NaN, y: -Infinity, z: -Infinity }
      const halfWidth = canvasWidth / 2
      const halfHeight = canvasHeight / 2
      vector.x = vector.x * halfWidth + halfWidth
      vector.y = -(vector.y * halfHeight) + halfHeight
      console.log(vector) // { x: NaN, y: -Infinity, z: -Infinity }
    },
    [camera, canvasWidth, canvasHeight]
  )

[granmoe]

For now, I'm just using a setTimeout within a useLayoutEffect as a workaround.

[drcmda]

They should be available after the first render so it indicates a bug, I'll look into it

[drcmda]

i tried to figure this out but hit a snag, it's solved - so it seems, but then some other thing breaks (heads up display). need to dig deeper.

[drcmda]

im not sure anymore if this IS a bug, everything seems proper, the cam seems to have correct data, size and viewport have data as well. i've found this: https://stackoverflow.com/questions/37098144/three-js-vector-project-gives-the-value-infinity and people are saying it should be infinite or that it should be unproject instead of project. could you read through this as i dont really understand the math.

[granmoe]

Sure, I will take a look and see what I can make of that

[drcmda]

so i read a little more (https://stackoverflow.com/questions/27409074/converting-3d-position-to-2d-screen-position-r69) and in some answer they used updateMatrixWorld, when i do this on the camera it works. i have no idea why it would be needed, but i have included it in the code that reacts to size changes. pushed a new 3.x beta for it, that should do the trick.

[drcmda]

edit: i get it now. i render the scene out via react first and start threejs rendering when everything's there. the renderer then proceeds to call camera.updateMatrixWorld(), which comes a frame later. this explains why it worked on timeout/raf.

[granmoe]

Ah, OK, so would it be correct to say that the three.js render is pushed onto the event queue (necessarily) after the first render of the React elements? I.e. it happens asychronously after the first React render? In that case, should there be a callout in the docs about this? Another option would be to create a hook that waits one tick the first time it's called, but that would probably be overkill.

[drcmda]

i think its fine now, it just calls updatematrix and that should be the end of it. but i will keep this in mind, i should maybe cause a single synchroneous (!) render frame on size changes (including the first run) so that what the scene-tree has rendered, threejs can render before we go into the effects of that scene tree.

i guess this is what @joelteply wanted to solve here: https://github.com/react-spring/react-three-fiber/pull/183