Blenderproc: Execute modules inside other modules

[SchmidtiT]

Hello together,

let me explain what I mean by that caption.
I want to randomise the camera position as well as the lighting. I have already taken a look at both examples: camera_sampling and light_sampling but I couldn't figure out how to execute the modules the way I intended.
At first I want to go to the first camera-position and randomise the lighting e.g. 10 iterations and then move on to the next camera-position and do the same thing again and again. So lets say I also have 10 camera-positions I have 100 different images at the end.
Is there a possibility to do that and if so can it also be done with 3 or more modules?

Thank you very much for your answer!

[themasterlink]

Hey,

there are different ways on how to solve this. But, first let me ask is it necessary for your test to have a fixed camera and changing light position? Else, why not change both at the same time the more variation the better as it expands the training space making it easier to generalize to your test data.

Assuming you really need this kind of data, because you want to have a fixed camera pose and a changing light for a specific scenario. You could design your own module and get inspired by the CameraObjectSampler, here the camera is also sampled after the objects are placed and this process is repeated several times.

But, keep in mind that BlenderProc 1.X is designed to create a mostly static scene with changing camera poses and that each run of BlenderProc creates its own static scene, as shown here:

@cornerfarmer Can the CameraObjectSampler also be used here, the documentation for the CameraObjectSampler has to be improved :D

Best,
Max

[cornerfarmer]

The CameraObjectSampler cannot be directly used here, as it does things the other way around: Each frame gets a new cam pose, but object poses stay for multiple frames. Also the LightSampler module only samples new lights once and does not sample different light poses for each frame.

So I would propose to use only the camera pose sampler inside your custom module and create the light poses via the Light API. Here is a short sketch of how the run method of your module could look like:

from src.utility.LightUtility import Light

def run(self):
    # Create a point light
    light = Light()
    light.set_type("POINT")
    light.set_location([5, -5, 5])
    light.set_energy(1000)

    # Get camera
    cam_ob = bpy.context.scene.camera

    for frame in range(self.total_frames):
        if frame % self.lightposes_per_camera == 0:
            # Sample new camera pose
            self._camera_pose_sampler.run()
        else:
             # Use same camera pose as in last frame
            cam_ob.keyframe_insert(data_path='location', frame=frame)
            cam_ob.keyframe_insert(data_path='rotation_euler', frame=frame)
            # Increase total frame counter
            bpy.context.scene.frame_end += 1

        # Sample new light pose
        light.set_location(self.sample_location(), frame=frame)

This assumes self._camera_pose_sampler to be your camera pose sampler to like it is done in the CameraObjectSampler and self.sample_location to be a method that samples a new light location. In your example self.total_frames would be 100 and self.lightposes_per_camera would be 10.

Let us know, if you need any further help

[SchmidtiT]

Wow thank you very much guys!

I think I experienced the Problem @cornerfarmer mentioned. I had the cameraSampler and lightSampler module both in my config file but when executing the program I experienced that only the camera pose was different in each image and not the lighting.
That's why I opened this Issue in the first place...

As a next step I will take a closer look on making my custom module with the camera pose sampler and the Light API if I get in any trouble I will comment here again if that is okay for you.

Again thank you very much! That helped a lot

[SchmidtiT]

@themasterlink I tried the method you suggested with the rerun.py program.
For that I deleted BlenderProc and reinstalled the newest version. (Had to delete it because I had an older version where rerun.py wasn't available)

But now every time I try to execute an example an Error is thrown:
Error: script failed, file: '/Users/user/BlenderProc/src/run.py', exiting.

The whole Traceback is as follows:
Traceback (most recent call last): File "/Users/user/BlenderProc/src/run.py", line 35, in <module> SetupUtility.setup_pip([]) File "/Users/user/BlenderProc/src/utility/SetupUtility.py", line 107, in setup_pip SetupUtility._ensure_pip(python_bin, packages_path, pre_python_package_path) File "/Users/user/BlenderProc/src/utility/SetupUtility.py", line 154, in _ensure_pip subprocess.Popen([python_bin, "-m", "ensurepip"], env=dict(os.environ, PYTHONPATH="")).wait() File "/Users/user/blender/Blender.app/Contents/Resources/2.91/python/lib/python3.7/subprocess.py", line 800, in __init__ restore_signals, start_new_session) File "/Users/user/blender/Blender.app/Contents/Resources/2.91/python/lib/python3.7/subprocess.py", line 1551, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/Users/user/blender/Blender.app/Contents/Resources/Contents/Resources/2.91/python/bin/python3.7m': '/Users/user/blender/Blender.app/Contents/Resources/Contents/Resources/2.91/python/bin/python3.7m'

I think the Problem is with the Blender Path:
It says: '/Users/user/blender/Blender.app/Contents/Resources/Contents/Resources/2.91/python/bin/python3.7m'

But the right Path is: '/Users/user/blender/Blender.app/Contents/Resources/2.91/python/bin/python3.7m'

I tried everything but could not find a solution for this.

Do you know how I can solve this?
Thank you very much!

[themasterlink]

Hey,

sorry we forgotten to test this on Mac OS, I just fixed it on my own machine, here is how you can fix it, too:

First you have to delete the old blender 2.91:

rm -rf /Users/user/blender/

Then after restarting the script, blender will be downloaded again and a new error pops up, you can fix that one quite simple, by removing "Contents", "Resources" from these four lines:

https://github.com/DLR-RM/BlenderProc/blob/ead97d373e82f80b70cac9f92fa4ff5ccc60295f/src/utility/SetupUtility.py#L94-L97

This should like this in the end:

            python_bin_folder = os.path.join(blender_path, major_version, "python", "bin")
            python_bin = os.path.join(python_bin_folder, "python3.7m")
            packages_path = os.path.abspath(os.path.join(blender_path,  "custom-python-packages"))
            pre_python_package_path = os.path.join(blender_path, major_version, "python", "lib", "python3.7", "site-packages")

PS: Blender does not support gpu rendering on Mac OS, so I would recommend to increase the amount of cpu threads used.

https://dlr-rm.github.io/BlenderProc/src.renderer.RendererInterface.html#src.renderer.RendererInterface.RendererInterface

This might then look like this in your Render Module config:

    {
      "module": "renderer.RgbRenderer",
      "config": {
        "output_key": "colors",
        "samples": 350,
        "render_normals": True,
        "normals_output_key": "normals",
        "render_distance": True,
        "distance_output_key": "distance",
        "cpu_threads": 12
      }
    },

I hope this helps. Tell me if it does it then we will release a patch for the current version.

Best,
Max

[SchmidtiT]

Max you are the best!

Thank you it worked!

[themasterlink]

This bug is now fixed with 1.11.1. Thanks for your help!