Pytorch3d: What `dist` in rasterization for?

[tomguluson92]

Hi @nikhilaravi,

I am wonder the practical meaning of dist in the following screenshots.

Since I assum z_buf represents depth of each intersection of one pixel, bary_coords represents barycentric coordinates for each intersection.

The explanation of pix_dists: giving the signed Euclidean distance (in NDC units) in the x/y plane of each point closest to the pixel. really make me confused. Could you please introduce with a little bit intuitive.
Looking forward to your reply, thank you very much~

[nikhilaravi]

Hi @tomguluson92 please refer to the PyTorch3D documentation where we have explained the input/output arguments of all the functions in detail. For this case:

If pix_to_face[n, y, x, k] = f then pix_dists[n, y, x, k] is the squared distance between the pixel (y, x) and > the face given by vertices face_verts[f]. Pixels hit with fewer than faces_per_pixel are padded with -1.

We calculate the squared euclidean distance from the centre of the pixel to the face edges. This is used subsequently in the soft blending functions. The intuition is that in rendering, there is a discrete step when deciding whether or not a face overlaps with a pixel in 2D. The diagram below illustrates this step change which is not differentiable.

To overcome this, we adopt the approach proposed in SoftRasterizer:
1) Introduce a small blur radius around each triangle face when rasterizing:

2) For each pixel, after identifying that a face overlaps with it, calculate and save the 2D euclidean distance from the center of the pixel to the face. Pixels which fall within the face have a negative distance and pixels which fall outside the face (in the blur region) have a positive distance.

3) When blending, apply a sigmoid to the 2D distance values of the form
prob = sigmoid(-dists/σ) where σ is a parameter which controls the width of the sigmoid. This creates a smooth boundary for the triangle instead of a hard boundary. Pixels inside
the face are mapped to the range of (0.5, 1) so have a high probability, and pixels which fall outside the face are mapped to the range (0, 0.5) so have a lower probability. This is an approximation of a binary mask where the threshold is at 0.5. The sharpness of the boundary can be varied by changing σ:
smaller values lead to a sharper distribution and larger values result in a more blurry boundary.
As σ → 0, the the probability map converges to the exact shape of the triangle.

[tomguluson92]

Thanks for your kindness! I think the intersection point of the kth near plane is a more intuitive usage than a pixel in this part?

For each pixel, after identifying that a face overlaps with it, calculate and save the 2D euclidean distance from the center of the pixel to the face. Pixels which fall within the face have a negative distance and pixels which fall outside the face (in the blur region) have a positive distance.