Meshroom: Alicevision/Meshroom inside blender

[argotan]

Have you thought about porting it to integrate it into blender ?, as meshroom is made with python and blender also has a python api. I think it would be very quick for them to adapt. And especially with the new version of blender 2.80, with the workspaces. You could make a specific workspace for Alicevision and Meshroom

[fabiencastan]

It sounds a bit difficult as the Meshroom graph and the blender graph are quite different.
I'm not sure what kind of interactions between Meshroom and Blender would be useful for graphic artists. It could probably be interesting on the mesh post-processing step, retopology and re-texturing.
It is not in our roadmap as we use zbrush and maya for that but I would be curious to follow the topic if someone is interested to investigate it.

The first interesting step would be to have a blender equivalent to our MeshroomMaya plugin which enables to load the Alembic reconstruction, setup image planes and provide a custom widget to visualize and navigate in the 3D reconstruction.

[tibicen]

I will take a look into your maya plugin and try to make a blender plugin port.

[DevoStevo66]

I've imported several Meshroom .obj files into Blender for retopology etc. Just import as .obj file from the Texturing Folder inside Meshroom folder created when file saved and completed. Just make sure the .mtl file and the .png texture images are in the same folder where you import the file into blender from and it should work (Blender 2.79).

[lilithebowman]

@DevoStevo66 is right! Imports straight into Blender with no issues! I was able to crop the vertices to just the ones I want and the model looks great! :)

[jonbragado]

Hi,
Follow up on this blender port plugin. Tibicen, have you been able to write the blender plugin for it? How did that go. I'm very interested in being able to do photogrammetry in blender.
Thanks.

[tibicen]

Hi @jonbragado ,
I was not thinking about porting it straight to blender. There was already port of SFM BlenderSFM.
I was thinking of importing work data straight to blender. As there was some changes in blender api, i was waiting a little with the addon. But now started to work on that. There is already excelent Point Cloud Visualiser . But it lacks .ply ASCII format which Meshroom use instead of alembic (.abc).
I wrote to the developer of Point Cloud Visualiser to help him add ASCII format (already did in my scripts, but it is better to team up in this kind of similarity).

I've also made base addon where you will point to your meshroom file, and it will extract all necesary data for you:

• Sparse point cloud
• Cameras with undisorded images
• Dense point cloud (optional)

As Blender imports alembic files, but it doesn't recognize meshroom alembic format, and crashes blender, i would rather go into ply format, so one would need to change output format od Sparse Reconstruction from .abc to .ply. [EDIT: Blender doesn't support vertex color, and probably this breaks the import of alembic files, but there could be more into it.]

@fabiencastan Can you tell me if your alembic files are in some kind of standard of is it just a database? Wonder if I can import it with blender native alembic importer.
No python api, except:

alembic_import()
bpy.ops.wm.alembic_import(filepath="", check_existing=True, filter_blender=False, filter_backup=False, filter_image=False, filter_movie=False, filter_python=False, filter_font=False, filter_sound=False, filter_text=False, filter_btx=False, filter_collada=False, filter_alembic=True, filter_folder=True, filter_blenlib=False, filemode=8, display_type='DEFAULT', sort_method='FILE_SORT_ALPHA', scale=1, set_frame_range=True, validate_meshes=False, is_sequence=False, as_background_job=False)
Load an Alembic archive

[tibicen]

As for further integration:

• Meshroom node structure would be very easy to implement into Blender nodes (input, output)
• it would work only as a UI which would execute alicevision cmdline interface as separate process.

[fabiencastan]

@tibicen PLY is a standard file format for point cloud.
But it doesn't contain camera information, visibility or link to input images names to retrieve undistorted images.

I would really recommend to use the ABC file format inside Blender and find someone with the blender knowledge to make the needed bug fix on ABC load. We have used Alembic especially for interoperability for other 3D softwares, so that would be sad to stay stuck due to this bug.

[simogasp]

Could it be the problem the fact that blender only supports hdf5 format for alembic while we use Ogawa?

[tibicen]

I wil try to ask around in the community Blender Alembic
For now I did some quick addon (working concept) which:

• imports cameras (images TODO, fov TODO, still wondering if I should place them in 3d or as camera background, different posibilities)
• imports sparse reconstruction with color and adjustable size and quantity (thanks to Jakub Uhlik)
• imports dense mesh obj (default False)
• imports textured obj

For now it goes only for _1 nodes, so as i assume only the one in first iteration... :/, later probably I should think of 2steps import (after reading the file it could iterate through nodes and import only the files that exist.

I will test it a little, add some assert, and close some empty paths, but it looks promising.

As for images: It is probably best to use undisorted ones for this?

[tibicen]

meshroom2blender I consider it still as a sketch, but it works in basic environment (one camera type, simple node tree).
I've got few questions to you guys:

1. In blender there are fixed for scene render, and that means all cameras are fixed. In another addon for camera imports guy solves it by using "active camera" and buttons (previous, next) for iterating through cameras and updating resolutions. I could implement those buttons when there would be more camera widths, and heights in cameras.sfm. Don't know if you maybe had similar issue in Maya?

[jonbragado]

Well this is really great the face that that the whole solved setup can be
imported directly into blender via your plugin.
thinking about the image plane, it would be great if they would behave like
in maya. where you can actually see it in 3d as an image plane and not a
textured card constraint to the frustum of the camera.

Does the export undistort the images? It will be great to see if there a
way to deal with distortion in the blender viewport

El dom., 10 feb. 2019 a las 13:55, Dawid Huczyński (<
[email protected]>) escribió:

meshroom2blender https://github.com/tibicen/meshroom2blender I consider
it still as a sketch, but it works in basic environment (one camera type,
simple node tree).
I've got few questions to you guys:

1. In blender there are fixed for scene render, and that means all
   cameras are fixed. In another addon for camera imports guy solves it by
   using "active camera" and buttons (previous, next) for iterating through
   cameras and updating resolutions. I could implement those buttons when
   there would be more camera widths, and heights in cameras.sfm.
2. didn't check vertical and horisontal alignments in one photo batch
   yet.
   [image: meshroom2blender]
   https://user-images.githubusercontent.com/5797670/52540173-8874dc80-2d7e-11e9-9a11-21fb34aba114.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alicevision/meshroom/issues/130#issuecomment-462178343,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtJRzFLEIOslwzry1prgaTIsedSZ6eLsks5vMJVpgaJpZM4UfFBQ
.

--
Jon Bragado
Lead Layout Camera TD
Dirección Técnica de Escenográfia
Method Studios

Linked-In: https://ca.linkedin.com/in/jon-bragado-diez-34b3bb42

-

[fabiencastan]

The node "ExportMaya" export the undistorted images. This node has nothing dedicated to Maya but was used to import the data into our MeshroomMaya plugin. But obviously, you should use the same to export to Blender.

[tibicen]

1. @jonbragado Sure, I was wondering about pros and cons of each option, but I can add image obj straight to the scene easily. How it is done in Maya (i mean placement)? Is the image placed as camera matrix? or additionaly "moved" away from camera and scaled accordingly to fit the camera lens (as projected from the camera point)?
2. I did put a undistorted images option to the plugin, but didn´t had time to test if it works in different examples. Most of the time my importer reads *.mg file and checks where files are stored, then imports them. So for undistorted images it uses *.exr from PrepareDenseScene output folder. (I hope this is ok?). I will check ExportMaya node too, maybe it will be easier to maintain one code if it works for both.

[jonbragado]

1- The image plane (IP) in maya is bound to the confines of the frustum.
Its always perpendicular to the "lens" sort to speak, or to its optical
axis. There isn't a scaling factor it has to be a depth factor. We should
be able to push the IP further from the camera but always making sure it is
attached to the limits of the frustum cone. It not one case the aspect
ratio of the image should change at all by scaling of any kind. It has to
be pushed units in distance away from the camera (ft or cm).

2- Distortion: As a tracking artist by trade there is a dicotomy here.

• Viewport Lens Distortion: WIthout a doubt something that would be the
  most efficient way to handle this complex world of distortion. Say we track
  a shot and export the data from a software. We export the distortion
  parameters in a .lens file and when we import the data in blender
  distortion is read from that file and applied to the viewport to undistort
  it live. That way we automatically are seeing in the viewport the CG with a
  correct undistorted plate and its a one to one of what we get.
  -

El mar., 12 feb. 2019 a las 2:48, Dawid Huczyński ([e-mail])
escribió:

>

1. @jonbragado https://github.com/jonbragado Sure, I was wondering
   about pros and cons of each option, but I can add image obj straight to the
   scene easily. How it is done in Maya (i mean placement)? Is the image
   placed as camera matrix? or additionaly "moved" away from camera and scaled
   accordingly to fit the camera lens (as projected from the camera point)?
2. I did put a undistorted images option to the plugin, but didn´t had
   time to test if it works in different examples. Most of the time my
   importer reads *.mg file and checks where files are stored, then imports
   them. So for undistorted images it uses *.exr from PrepareDenseScene
   output folder. (I hope this is ok?). I will check ExportMaya node too,
   maybe it will be easier to maintain one code if it works for both.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alicevision/meshroom/issues/130#issuecomment-462711418,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtJRzIDcpsb0M7mAxWeyxH2UHi3RPz9Dks5vMpvigaJpZM4UfFBQ
.

--
Jon Bragado
Lead Layout Camera TD
Dirección Técnica de Escenográfia
Method Studios

Linked-In: https://ca.linkedin.com/in/jon-bragado-diez-34b3bb42

-

[jonbragado]

• UD plate render: We would have to undistort the plate, render it, and
  then then assign those UD plates to the camera. It takes up a lot unneeded
  space on disk. Say we found the distortion isnt right, and we change it.
  Well that means that we would have to rerender all the plates again UD'd
  and then reapplying it back. Overscan becomes key here. Most software can
  handle undistortion but the black edge, the overscan isn't right. It's
  always cropped, not only that but the frame size doesnt change, which is
  completely wrong. a UD plate should always be bigger that the original
  distorted plate.

Frankly the previous Viewport Lens Dostortion approach is the way to go, it
is trickier to engineer, probably but in the long run it sets up Blender
for a workflow that allows very fast back and forth and reduces disk space.
WIth the Viewport Lens Distortion, you could even set a toggle button to
playblast with the distortion applied back so that your blasts would be
seen correctly as they are bieng rendered!

By no means im saying this would be easy, but with the new viewport EEvEE
im sure it can be done.

link:
https://pdfs.semanticscholar.org/941e/a54629a3322e5de9d1ad3d1705ef3b8fa1b4.pdf

El mar., 12 feb. 2019 a las 11:45, Jon Bragado ([e-mail])
escribió:

1- The image plane (IP) in maya is bound to the confines of the frustum.
Its always perpendicular to the "lens" sort to speak, or to its optical
axis. There isn't a scaling factor it has to be a depth factor. We should
be able to push the IP further from the camera but always making sure it is
attached to the limits of the frustum cone. It not one case the aspect
ratio of the image should change at all by scaling of any kind. It has to
be pushed units in distance away from the camera (ft or cm).

2- Distortion: As a tracking artist by trade there is a dicotomy here.

• Viewport Lens Distortion: WIthout a doubt something that would be
  the most efficient way to handle this complex world of distortion. Say we
  track a shot and export the data from a software. We export the distortion
  parameters in a .lens file and when we import the data in blender
  distortion is read from that file and applied to the viewport to undistort
  it live. That way we automatically are seeing in the viewport the CG with a
  correct undistorted plate and its a one to one of what we get.
  -

El mar., 12 feb. 2019 a las 2:48, Dawid Huczyński (<
[email protected]>) escribió:

>

1. @jonbragado https://github.com/jonbragado Sure, I was wondering
   about pros and cons of each option, but I can add image obj straight to the
   scene easily. How it is done in Maya (i mean placement)? Is the image
   placed as camera matrix? or additionaly "moved" away from camera and scaled
   accordingly to fit the camera lens (as projected from the camera point)?
2. I did put a undistorted images option to the plugin, but didn´t
   had time to test if it works in different examples. Most of the time my
   importer reads *.mg file and checks where files are stored, then imports
   them. So for undistorted images it uses *.exr from PrepareDenseScene
   output folder. (I hope this is ok?). I will check ExportMaya node too,
   maybe it will be easier to maintain one code if it works for both.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alicevision/meshroom/issues/130#issuecomment-462711418,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtJRzIDcpsb0M7mAxWeyxH2UHi3RPz9Dks5vMpvigaJpZM4UfFBQ
.

--
Jon Bragado
Lead Layout Camera TD
Dirección Técnica de Escenográfia
Method Studios

Linked-In: https://ca.linkedin.com/in/jon-bragado-diez-34b3bb42

-

--
Jon Bragado
Lead Layout Camera TD
Dirección Técnica de Escenográfia
Method Studios

Linked-In: https://ca.linkedin.com/in/jon-bragado-diez-34b3bb42

-

[natowi]

Also see https://github.com/alicevision/meshroom/issues/460#issuecomment-520252376

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[natowi]

https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer now has the best support for Meshroom.