Three.js: unproject - How to get the real position on 'screen(top, left)' of specific 3d object?

[alivedise]

Upon examples, I knew there's some way to project one point on screen's (x, y) to '3d''s points. But I can't figure out how to translate it back. Thanks for any clue.

[fabricasapiens]

Probably using the Projector, but MrDoob obviously knows more about this :)

[mrdoob]

Uhm, maybe we should have a object3d.screen which would be a Vector2 and update it on every render?

[fabricasapiens]

If that object3D.screen Vector is only used to translate the object3D.position of every object3D to 2D, then would object3D.screen not be of too too little value? I mean: I think it would be more useful to be able to translate any point x,y,z of any mesh at position x,y,z to 2D coordinates.

That in turn means it would never be possible to precalculate every 3D position of every object.

It would imho be useful to have for example a function THREE.3dto2d() that takes either a Vector3 (representing a point in the scene) or a Vector3 and an object (where the Vector3 represents a point on the object).
The function then calculates the 3D position in the scene.
At last, it takes the camera, its position, rotation, the projectionMatrix etc, and translates the point back to 2D.

I guess the camera should be a parameter of the function as well...

[mrdoob]

Well... Projector.projectScene kinda does that already ;)

[fabricasapiens]

See, I knew it was the Projector... ;-)

Is there an example using it?

[mrdoob]

Hmm... not yet... those functions are designed to be used with the Renderer... they return a list of elements to render with no relation to the original objects. Maybe I should just put some link to the original objects and that'll do... Hmmm, need to think that a bit more.

I'm currently on holidays, but I'll do a lens flare demo (or something) as soon as I get back (next week).

[fabricasapiens]

Sure, enjoy your holidays and I'm looking forward to the new demo! :-)

[fuzzthink]

Hmm, this used to work for me (r32), doesn't anymore in r35..

  toScreenXY: function(position, camera, jqdiv) {
    // converts 3D position to screen coords
    var pos = position.clone();
    projScreenMat = new THREE.Matrix4();
    projScreenMat.multiply( camera.projectionMatrix, camera.matrix);
    projScreenMat.multiplyVector3(pos);
    return { x: (pos.x+1)*jqdiv.width()/2 + jqdiv.offset().left, 
             y:(-pos.y+1)*jqdiv.height()/2 +jqdiv.offset().top }; 
  },

pos after projScreenMat.multiplyVector3(pos); used to be capped between -1 and 1 ( (0,0) at center of view.), but now seems to vary unexpectedly..

[mrdoob]

Yeah, camera.matrix needs to be replaced now with camera.matrixWorldInverse.
Try with this:

toScreenXY: function ( position, camera, jqdiv ) {

    var pos = position.clone();
    projScreenMat = new THREE.Matrix4();
    projScreenMat.multiply( camera.projectionMatrix, camera.matrixWorldInverse );
    projScreenMat.multiplyVector3( pos );

    return { x: ( pos.x + 1 ) * jqdiv.width() / 2 + jqdiv.offset().left,
         y: ( - pos.y + 1) * jqdiv.height() / 2 + jqdiv.offset().top };

}

[fuzzthink]

Yup that worked, didn't know about the matrices changes. Hope this projection function answered alivedise's question.

[imrantariq2011]

I think it is the right thread to ask my question regarding 3D Cube. I have a cube drawn on canvas with clickable faces. The cube is freely rotating on X axis and Y axis. What I want is to get the most front face when the user release the mouse and make it fully visible by rotating automatically. Any help in this context would be highly appreciative.

Thank you.

[mrdoob]

@imrantariq2011: That's not really related to this issue, do you mind opening a new one? Thanks.

[ted-dunstone]

For texture mapping purposes I'd like to be able to get the x,y screen coordinates for each vertice for a given face.

I can see how this could be done using the above (toScreenXY: function) - but first the world transformed (scaled, translated and rotated) vertices for a given face are required. I've spent some time looking at Projector.projectScene and I wonder if there is a simpler way of achieving the above goal and it looks like object.matrixWorld.getPosition() might get the centroid of the object rather than the face vertices.

[mrdoob]

Yeah, the code above was for a one of. If you need to project many points if best to exctract the bits of code from .projectScene() itself.

[arieljake]

people who get to here may want to know about this:

https://github.com/zachberry/threejs-tracking-3d-to-2d

[arieljake]

I use this, it works:

function toScreenXY(pos3D)
        {
            var projector = new THREE.Projector();
            var v = projector.projectVector(pos3D, camera);
            var percX = (v.x + 1) / 2;
            var percY = (-v.y + 1) / 2;
            var left = percX * window.innerWidth;
            var top = percY * window.innerHeight;

            return new THREE.Vector2(left, top);
        }

On Friday, August 1, 2014 11:24 AM, Artem Fitiskin [email protected] wrote:

When i use toScreenXY, i'm get very big values.
console.log(pos): // x: -1474.1436403989792, y: -730.829023361206, z: 3.0004000663757324
and after multiply it with jqdiv, function returns very big x and y.
What i'm doing wrong?
—
Reply to this email directly or view it on GitHub.

[WestLangley]

@arieljake

As stated in the guidelines, help requests should be directed to stackoverflow. This board is for bugs and feature requests.

[arieljake]

fine...yet mr doob himself put code, so what gives?

[WestLangley]

@arieljake #1979

[craftfortress]

Incase anyone still needs this after upgrading...

function screenXY(obj){

  var vector = obj.clone();
  var windowWidth = window.innerWidth;
  var minWidth = 1280;

  if(windowWidth < minWidth) {
    windowWidth = minWidth;
  }

  var widthHalf = (windowWidth/2);
  var heightHalf = (window.innerHeight/2);

  vector.project(camera);

  vector.x = ( vector.x * widthHalf ) + widthHalf;
  vector.y = - ( vector.y * heightHalf ) + heightHalf;
  vector.z = 0;

  return vector;

};

[HjalmarSnoep]

Ok, very old issue, but I had a nested object, which didn't work with the above code.
My case was: I wanted the real x-and-y (and z) position for a nested object after rendering, so I adapted the above examples.
Please note:

1. You now give the whole object as an argument instead of only the object.position in the above examples.
2. Also this will only work after the renderer has done it's job and called
3. It also works for non-nested objects, but it will do a few calculations too many.
4. It also gives you a screen-Z, which can be used to sort domElements or zSort labels.
   The screen-z relative is between 0 and 1, relative to the front and back clipping planes of the camera/renderer and this works fine for my purposes (sorting).
5. You can get the renderers current width and height by renderer.domElement.width and renderer.domElement.height.

scene.updateMatrixWorld(); 

and

parent.updateMatrixWorld(); 

``` js
function nestedObjecttoScreenXYZ(obj,camera,width,height)
{
var vector = new THREE.Vector3();
vector.setFromMatrixPosition( obj.matrixWorld );
var widthHalf = (width/2);
var heightHalf = (height/2);
vector.project(camera);
vector.x = ( vector.x * widthHalf ) + widthHalf;
vector.y = - ( vector.y * heightHalf ) + heightHalf;
return vector;
};

typical call:
```js
var screenpos=NestedObjecttoScreenXY(object,camera,renderer.domElement.width,renderer.domElement.height,true);

Just in case anyone runs into the same problem.