Pytorch3d: Non-square mesh rasterizer?

[wangg12]

🚀 Feature

Motivation

Pitch

NOTE: we only consider adding new features if they are useful for many users.

[nikhilaravi]

@wangg12 Thank you for raising this issue!

Rendering of non square images is on our roadmap. If you are interested in trying to implement this feature, please update this issue with a detailed outline of your plan and we can review it, following which you are welcome to submit a pull request.

[jbohnslav]

I'm also quite interested in this feature!

[nikhilaravi]

@jbohnslav We are planning to implement this soon.

[czkg]

This feature is critical!

[bhadresh74]

Hi @nikhilaravi , I am also interested in this feature.
Do you have any approximate time frame when this feature will be made available?

[nikhilaravi]

@bhadresh74 we are currently working on this feature!

[soubhiksanyal]

Hi @nikhilaravi, I am also interested in this particular feature. I will give it try for a pure optimisation based method I'm working on. Is there any approximate time frame for this?

[nikhilaravi]

Hi @soubhiksanyal! :) It certainly looks like this is a popular feature request :) We have been working on this feature and will aim to have it ready in the next few weeks. I'll update this issue with a more accurate time estimate soon.

[samxuxiang]

Any update on this?

[luoyuxuan0528]

Any update on this?

@nikhilaravi I am new to pytorch3d, and this issue make me wonder whether it is the best choice.

[gkioxari]

Hi @luoyuxuan0528
As Nikhila has pointed out we are working on this feature. We are only a few people working on this library and extending it with new functionality and open-sourcing it to the public to serve you. So please be understanding and kind.

[luoyuxuan0528]

Hi @luoyuxuan0528
As Nikhila has pointed out we are working on this feature. We are only a few people working on this library and extending it with new functionality and open-sourcing it to the public to serve you. So please be understanding and kind.

Sorry for that. Looking forward to this feature! @gkioxari @nikhilaravi

[jbohnslav]

Hi all, do you have an internal PR / plan for this issue you'd be willing to share? I'm interested in contributing to this issue, as it is essential for my project.

[gkioxari]

This feature will be added by end of year!

[jbohnslav]

Great, thank you @gkioxari! If you need any help / work done, I'd be happy to

[nikhilaravi]

@jbohnslav @luoyuxuan0528 @samxuxiang @soubhiksanyal @bhadresh74 @czkg @wangg12 what are your use cases for non square image rasterization? e.g. do you want to vary the number of pixels in X and Y but maintain square pixels?

We will be releasing an implementation shortly and want to check that it meets your use cases.

[bhadresh74]

@nikhilaravi
Yes, precisely that.
Thanks for checking in with us.

Quick question, are you also planning to add new class to handle camera perspective to allow non-square image for rasterization?

[jbohnslav]

Yes, I want to do exactly that. I'm working with real cameras, which have square pixels but rectangular sensors.

[nikhilaravi]

@bhadresh74

are you also planning to add new class to handle camera perspective to allow non-square image for rasterization?

We are not changing the cameras. If you want to use non square images with square pixels you don't need to change any of the camera parameters.

[nikhilaravi]

This feature is now available in PyTorch3D for pointcloud and mesh rasterization. Please read this note for details on how to set the (H, W) correctly for rasterizing non square images.