Open3d: Determining the Proper Camera Position

[Zumbalamambo]

I would like to position the camera based on the homogeneous matrix. Following is my homogeneous matrix,

[[ 9.01101806e-01 -5.01196868e-03  4.33578615e-01 -1.34760000e+02]
 [ 1.61232776e-01  9.32107647e-01 -3.24313623e-01  8.03920000e+01]
 [-4.02516493e-01  3.62146675e-01  8.40731978e-01  2.11000000e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]

How do I find the corresponding up vector, set_front,set_lookat, appropriate zoom values with these values? I used the following code to do so but it is still not there yet,

import open3d as o3d
import numpy as np


pcd = o3d.io.read_point_cloud("assets/pcd_sel.pcd")
vis = o3d.visualization.VisualizerWithKeyCallback()
vis.create_window()
vis.get_render_option().background_color = np.asarray([0, 0, 0])

view_ctl = vis.get_view_control()

vis.add_geometry(pcd)

view_ctl.set_up([-4.02516493e-01, 3.62146675e-01, 8.40731978e-01])
view_ctl.set_front([-4.02516493e-01, 3.62146675e-01, 8.40731978e-01])
view_ctl.set_lookat([-1.34760000e+02 ,8.03920000e+01 ,2.11000000e+00])

vis.run()
vis.destroy_window()

[griegler]

You could use something like this

cam = view_ctl.convert_to_pinhole_camera_parameters()
cam.extrinsic = T # where T is your matrix
view_ctl.ctr.convert_from_pinhole_camera_parameters(cam)

[Zumbalamambo]

@griegler thank you! I tried it. Following is my code,

import open3d as o3d
import numpy as np


pcd = o3d.io.read_point_cloud("assets/pointcloud.pcd")
vis = o3d.visualization.VisualizerWithKeyCallback()
vis.create_window()
vis.get_render_option().background_color = np.asarray([0, 0, 0])

view_ctl = vis.get_view_control()

vis.add_geometry(pcd)
pose = np.array([ 0.23522,-7.0289e-17,-0.97194,-129.54,
0.97194,-7.3988e-17,0.23522,59.464,
-8.8446e-17,-1,5.0913e-17,2.11,
0,0,0,1]).reshape(4,4)
cam = view_ctl.convert_to_pinhole_camera_parameters()
cam.extrinsic = pose # where T is your matrix
view_ctl.convert_from_pinhole_camera_parameters(cam)
vis.run()
vis.destroy_window()

Unfortunately, the camera is still not focussing on the right axis :( am I missing something?

[Zumbalamambo]

@griegler should I transpose the homogeneous matrix?

[griegler]

It depends on you transformation. T in the example snippet assumes that T = [R | t] # 4x4 with last row (0,0,0,1) where R is the rotation from world coordinate system to camera coordinate system and t is the translation from wcs to ccs.

[Zumbalamambo]

@griegler isn't there any other way to update the camera position? No matter what I do, it's always wrong... Been trying for a week...The last row is [0,0,0,1] but still the same problem :(

[Zumbalamambo]

@griegler Could you please suggest how can I retrieve the front_vector and up vector from my homogeneous matrix for open3d?

[griegler]

Where do you have this matrix from, or how has it been retrieved? Could you possibly share the data, and what it should look like?

[griegler]

And maybe this (first part) might help.

[Zumbalamambo]

Give @griegler a noble prize!!!!!!!... Thank you so much! that article helped me to the core!!!!!!!!!!

[Zumbalamambo]

The visualization looks so much better now... Now more flickering... I would like to ask one more doubt :)

This is my data,

-0.98543,9.5816e-17,0.17007,-135.8
-0.17007,-3.5129e-17,-0.98543,79.68
-8.8446e-17,-1,5.0913e-17,2.11
0,0,0,1

what must be the input for set_ftront? Should I use the forward vector? or the right vector? :) Also is there any mathematical formula to determine the threshold that I must set for set_zoom? or is it just the hardcoded value?

[Zumbalamambo]

@griegler should I do any change in the signs or the position of the set_front coordinates? The camera is moving backward across the trajectory instead of facing forward

[griegler]

If your transformation matrix T is

| R t |
| 0 1 |

then is R

| right.T   |
| -up.T     |
| -front.T  |

and t = -R eye

[Zumbalamambo]

@griegler it works like a charm! thank you...