Meshroom: [Feature Request] Support for 360 Camera images

[jjisnow]

It would be great if there was support for the standard flat 360 camera projection images given out by 360 cameras.

J

[natowi]

You could try Meshroom-2019.1.0\aliceVision\bin\aliceVision_utils_split360Images.exe (cli only)

[jjisnow]

Thanks! I've done that now, but there is no help in the app on computing the necessary metadata to use with meshroom

[natowi]

What do you mean with

there is no help in the app on computing the necessary metadata to use with meshroom

?
Do you need help with the cli or meshroom?

[jjisnow]

I figured it out thanks. The images split perfectly. When i tried mesh room, the sfm step failed with images from one panorama, but if i renamed the files from several panoramas to get some parallax in sequential images to get some initial point cloud, then augmented the scan with the remaining photos, then on dense cloud choose "Downscale" =1, as noted from #409 it proceeds perfectly!

[jeffreyianwilson]

I know it can split the images but does it deal with the cube map pinhole cameras as a fixed rig?

[natowi]

@jeffreyianwilson I have tested this with the Datasets from here and it works.

[Baasje85]

@natowi I would leave this issue open. And have the new openMVG code imported to support panoramas natively.

[jeffreyianwilson]

Excellent, processing hundreds if not thousands of panoramas into cube maps images is an unnecessary waste of storage

[jeffreyianwilson]

Does Meshroom/Alice Vision support camera rigs/fisheye lenses? I want to take the individual camera output from a 360 rig (8 x200degree cameras) and apply this rig per shot. The parallax offset is considerable and prevents close range precision when using Equirectangular (converted to cubemap) images

[Baasje85]

Typically such rig does not use fish eye lenses, but fixed focal lenses. If you would calibrate this rig (and this is the missing documentation part) this would be better than the combined image, more image detail, more overlap per photo and thus depth. Then again, openMVG recently showed that calibrated stitched images are superior to unstitched unrigged images with respect to matching them in SfM. So you may wonder if a workflow: start with pre-stitched then augment with raw images gives faster results.

[jeffreyianwilson]

The Insta 360 Pro 2 and Pro use 200 degree lenses. Like I said, close proximity features and camera offset from the nodal point prevent any sort of precision from baked equirectangular images.

[jeffreyianwilson]

I am looking at constructing a "calibration room" which would have enough features to treat each lens/sensor separately but as a whole as part of a rig.

[Baasje85]

@jeffreyianwilson you might be interested in https://blog.elphel.com/category/calibration/

[fabiencastan]

Hi @jeffreyianwilson,

Does Meshroom/Alice Vision support camera rigs/fisheye lenses? I want to take the individual camera output from a 360 rig (8 x200degree cameras) and apply this rig per shot.

Yes, this is fully supported as explained here:
https://github.com/alicevision/meshroom/wiki/Multi-Camera-Rig
The calibration of the rig is fully automatic.

Would you be open to share one of your datasets with me? I would be interested to do more tests on these setups. If yes, you could use the private mailing-list [email protected].

Thanks

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[CorentinLemaitre]

Hello, I have a Samsung gear 360 camera and i do 360 picture with 30 Megapixel equirectangular every 10 meter to survey bicycle routes. Then I add geolocation to pictures and share the pictures on Mapillary mostly to add map feature in OpenStreetMap.
I wonder if there is support of 360 pictures or it is still something to develop.
I could share any picture i have taken if it is helpful.

[fabiencastan]

@CorentinLemaitre,
Yes, it would be interesting to have access to a dataset made with the Samsung Gear 360.

There is no support for 360° images in input. We have support for a rig of synchronized cameras, but I don't know if you have access to the raw images on the Samsung Gear 360 (before stitching).

[CorentinLemaitre]

I have 360 before processing because this camera (2016) don't do the stitching. After I have done the stitching process I delete these files. I have really few that left on my computer.
Here is an example of the picture i have before stitching :

And the result after stitching :

[EwoutH]

I have a small dataset of closely located 360-degree equirectangular images (taken with a Gear 360 2016). I previously used them with Cupix. I can provide one (in private) if it helps development.

Here are five images from my old rooftop to start with:

Unstitched (7776x3888 dual fisheye)

• 360_0497
• 360_0498
• 360_0499
• 360_0500
• 360_0501

Stitched (7776x3888 equirectangular)

• SAM_100_0497
• SAM_100_0498
• SAM_100_0499
• SAM_100_0500
• SAM_100_0501

[fabiencastan]

Thanks for the datasets.

[Baasje85]

@fabiencastan would you be interested in other vendors too?

[natowi]

@Baasje85 I think it would not hurt to have a few different datasets for testing.

@fabiencastan We could use a demo&testing dataset similar to https://github.com/alicevision/dataset_monstree Maybe we can put something together based on user contributions for a few different camera models.

[SM-26]

@Baasje85 I think it would not hurt to have a few different datasets for testing.

@fabiencastan We could use a demo&testing dataset similar to https://github.com/alicevision/dataset_monstree Maybe we can put something together based on user contributions for a few different camera models.

I'll be more than happy to help
I have an Insta360 One X

any notes or pointers on how you want a sample set?
how may pictures? HDR on or off? indoor or outdoor?

[tscibilia]

Here's my contribution... 5 image interior dataset from an Insta360 OneX

I actually want to use meshroom for interiors so I have a lot more if it's helpful (an entire house). I could provide it privately from github, just contact me.

[natowi]

I´m merging the shared datasets into one repository with a hand full of images per dataset, all under CC-BY-SA-4.0 license. If you are ok with it, leave a thumbs up on this post and I´ll add your dataset.
@EwoutH @Baasje85 @SM-26 @tscibilia

When it is well structured, I can move it to AliceVision. https://github.com/natowi/meshroom-360-datasets

[SM-26]

tscibilia beat me to the punch.
But, I saw that there is no info about the Insta360 one X on the camera DB

Sensor: 1/2.3" (~6.16 x 4.62mm)
source

[natowi]

@SM-26 what is the _make_ and _model_ in the metadata?

how may pictures? HDR on or off? indoor or outdoor?

We don´t need too many images (let´s say images from ~6 different locations), these datasets are just for testing and demonstration.
I think indoor/outdoor with and without HDR would be nice. If you are using a tripod, you could use the same positions for HDR on/off.

[SM-26]

@SM-26 what is the _make_ and _model_ in the metadata?

how may pictures? HDR on or off? indoor or outdoor?

We don´t need too many images (let´s say images from ~6 different locations), these datasets are just for testing and demonstration.
I think indoor/outdoor with and without HDR would be nice. If you are using a tripod, you could use the same positions for HDR on/off.

camera brand: Arashi Vision
camera model: Insta360 ONE X

I'm on it, good thing the weekend is here.

[SM-26]

@SM-26 what is the _make_ and _model_ in the metadata?

how may pictures? HDR on or off? indoor or outdoor?

We don´t need too many images (let´s say images from ~6 different locations), these datasets are just for testing and demonstration.
I think indoor/outdoor with and without HDR would be nice. If you are using a tripod, you could use the same positions for HDR on/off.

Sorry it took me such a long time.
I've created a PR
I'd love to help as much as I can

[tscibilia]

Just catching up, I saw the repo and @SM-26 pull request so I did a PR of my own