I recently tweeted mrdoob about this, but I think this may be a better forum to express this issue.
I would like for there to be some way to set vanishing points in the image that the cubes I add to a scene adjust to. Like below:

The cubes that I add onto a scene now using scene.add(cube)
assume that the center of the screen is the vanishing point and only assume one vanishing point, unlike above. I can certainly adjust the position of the camera to simulate an alternate vanishing point, but is there a way for me to set the vanishing point explicitly? More importantly, is it possible to have multiple vanishing points?
Note, BTW, that vanishing points don't have to be inside the scope of the image (or the web page, in this instance)
Why not just set the camera.position, camera.lookAt and Field of View?
Effective editing of these three variables allows you to place the (1, 2 or 3) vanishing points wherever you wish...
Can you please give an example of setting two vanishing points in code? This would be much appreciated.
@theo-armour I think he means a way of doing it directly with code, without trial an error.
Can't think of how it can be done myself, I lack the maths knowledge. Hopefully someone else has some pointers.
@aashidham
More importantly, is it possible to have multiple vanishing points?
Without hacking the camera.projectionMatrix code, in camera space the answer is no. However, as you said, in world space you can simluate it.
Can you explain again what capability you'd like to have?
Are you interested, for example, in being able to programmatically set the camera parameters, given the coordinates of the 1, 2, or 3 (world) axis vanishing points?
In other words, to ask the question another way, what parameters do you want to specify, and what parameters do you want to programmatically determine?
@aashidham
A typical reason for wanting to do this type of action is when somebody is
trying to overlay a 3D image over a photograph.
Is the the type of thing you are looking for?
Most helpful comment
@aashidham
A typical reason for wanting to do this type of action is when somebody is
trying to overlay a 3D image over a photograph.
Is the the type of thing you are looking for?