Blenderproc: blenderproc on videos

[soans1994]

hello authors,

i need help to use blenderproc to generate synthteic data from a video clip. i want to use the blenders camera tracking feature to replace a traffic sign board in a video, with my 3d traffic sign model and then generate generate training data. i think this is a good idea compared to procedurally modeling the traffic environment (lighting, shadows, roads and trees). Is this possbile?

thank you

[cornerfarmer]

Hey @soans1994

camera tracking is not supported by blender proc and this will probably also not change in the near future, as tracker placement and orientating the 3D object relative to the video is very hard to automate.
What are you trying to achieve in the end? If you want to generate training data for traffic sign detection, you might start with rendering your traffic signs on alpha background and then placing the render results on random backgrounds.

[soans1994]

@cornerfarmer
thank you for reply, as that is what i want.

1. does the blenderproc has examples of placing random backgrounds?
2. so genearating dataset from a video is not possible?

[cornerfarmer]

1. We recently added a new example for that which will be published via the next version. I uploaded you the relevant files, so you can already take a look at it: random_backgrounds.tar.gz. Just unzip it in your blenderproc root directory and then follow the README of the added example.
2. No, BlenderProc does not support videos yet. You could of course adjust the example I uploaded above to paste objects on videos instead of images.

[soans1994]

@cornerfarmer

thank you very much for your kind response.

[soans1994]

@cornerfarmer
hello, can i use obj format? the ply format doesnt save my model textures:(

[ideas-man]

@soans1994 Hi,

Yes, just make sure your .mtl file is of the right format. More info in this post.

[soans1994]

@ideas-man Thank you I will check it out