Pytorch3d: How to Render Vertex Colored mesh?

[wangsen1312]

❓ Questions on how to use PyTorch3d

Any simple example for render vertex colored .ply mesh?

[nikhilaravi]

@wangsen1312 You can initialize a mesh using vertex rgb colors as a texture and use that in the rendering pipeline e.g.

from pytorch3d.structures import Textures, Meshes

textures = Textures(verts_rgb=verts_rgb_colors)
mesh = Meshes(vers, faces, textures)

[wangsen1312]

@nikhilaravi Thank you for your response.
I have tried the code you offered and replace the tex with below:

verts_rgb_colors = torch.zeros([1, num_verts, 3]).to(device)
tex = Textures(verts_rgb=verts_rgb_colors)

But it occurs some error:

File "/home/xxz/Code/RNN/renderTexured/customshader.py", line 17, in forward
images = self.shader(fragments, meshes_world, *kwargs)
File "/home/xxz/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(input, **kwargs)
File "/home/xxz/anaconda3/lib/python3.7/site-packages/pytorch3d/renderer/mesh/shader.py", line 269, in forward
texels = interpolate_texture_map(fragments, meshes)
File "/home/xxz/anaconda3/lib/python3.7/site-packages/pytorch3d/renderer/mesh/texturing.py", line 43, in interpolate_texture_map
faces_uvs = meshes.textures.faces_uvs_packed()
File "/home/xxz/anaconda3/lib/python3.7/site-packages/pytorch3d/structures/textures.py", line 147, in faces_uvs_packed
return list_to_packed(self.faces_uvs_list())[0]
File "/home/xxz/anaconda3/lib/python3.7/site-packages/pytorch3d/structures/utils.py", line 116, in list_to_packed
N = len(x)
TypeError: object of type 'NoneType' has no len()

It seems no faces_uvs_list() during the render?

Any better solutions?

[wangsen1312]

@nikhilaravi Due to the shader problem, I found such HardPhongShader works well on vertex_rgb. Thank you very much~

[nikhilaravi]

@wangsen1312 Yes! Glad you resolved your issue!
There are different shaders for each texture type - as you probably noticed, the HardPhongShader calls a function interpolate_vertex_colors whereas the TexturedPhongShader calls a function interpolate_texture_map which expects the faces_uvs and verts_uvs.

[bhadresh74]

@nikhilaravi
On a similar note, do you have any plans to add a new feature like rendering face colors instead of vertex colors?
If no, can you just briefly explain if its doable in this repo?

[nikhilaravi]

@bhadresh74 we are working on adding support for texture atlassing i.e. an (R, R, 3) texture map per face in the mesh, where R is the texture resolution. Then rendering with face colors would involve setting (R=1) to get an (F, 1, 1, 3) map which is essentially a per face color.

[bhadresh74]

@nikhilaravi Thank you for adding this feature too. Excited to try that.

[nikhilaravi]

@bhadresh74 support for texture atlassing has now been added. Here is a complete example of how to load the textures as an atlas, create a mesh and and render it: https://github.com/facebookresearch/pytorch3d/blob/master/tests/test_render_meshes.py#L468.

[bhadresh74]

@nikhilaravi
That's great. Thank you for the update.