Three.js: SelectionBox: Add support for pixel-perfect selection.

[MaxmaxmaximusAWS]

P.S. it is not question, it is feature request for frustum.intersectsGeometry(mesh.geometry)

I mean not bounding box and bounding sphere, but exactly pixel perfect, I mean.

Maybe shader or special renderer? Give different colors to different meshes, and by rendered colors detect what objects were selected?

After three days of googling, I haven't found a single answer on the Internet.

For example, this Box should not be selected:

[Mugen87]

As mentioned at stackoverflow, this issue is about SelectionBox.

https://threejs.org/examples/misc_boxselection

I understand that you would like to see SelectionBox pixel perfect but I'm not sure the additional complexity and overhead would be acceptable. A PR would make it easier to evaluate this feature.

[MaxmaxmaximusAWS]

@Mugen87

How does Blender do it? If I select many objects, he clearly understands what I have selected. Do you have any ideas about their algorithm?

Maybe we can paint objects in the scene in different colors, and render the scene, and then use the pixel colors to understand what colors are visible? And so we exploit the parallel nature of the graphics card? Its like raycasting, but by every selection pixel.

P.S. Can we write a post processor shader that will understand where which object?

[gkjohnson]

Maybe we can paint objects in the scene in different colors, and render the scene, and then use the pixel colors to understand what colors are visible? And so we exploit the parallel nature of the graphics card? Its like raycasting, but by every selection pixel.

This would make selection asynchronous which might not be desirable. Copying render target data from the GPU to to the CPU can also be a slow operation and will scale based on canvas size / pixel density. GPU picking for raycasting is a bit more consistent because you only need to render / copy a single pixel.

How does Blender do it? If I select many objects, he clearly understands what I have selected. Do you have any ideas about their algorithm?

Blender may have extra underlying acceleration data structures to facilitate performant interactions like this that three.js doesn't provide out of the box. With simple geometry performance may not be all that bad though. For all the meshes you want to check selection for you could perform a triangle / selectionbox frustum intersection for every triangle in the mesh to see if it is selected. If the bounding box / sphere of a mesh is completely contained in the frustum you could skip the triangle check and just mark it as selected. Supporting all the corner cases of instanced, skinned, etc meshes may get complicated, though.

If performance turns out to be an issue you could only run the selection check when the user releases the mouse rather than on every frame. For particularly complex meshes you could use some kind of spatially indexed data structure like a quad tree or bvh which there are some libraries for.