Pcl: GPU Normal Estimation

[barnjamin]

Your Environment

• Operating System and version: Ubuntu 16.04
• Compiler: GCC 5.4.0 nvcc 9.0
• PCL Version: 1.8.1

Context

Trying to download computed Normals from the device array

Expected Behavior

pcl::PointCloud<PointNormal> p;
computed_normals.download(*p);

or something like that

Current Behavior

Only allowing PointXYZ or a std::vector containting PointXYZ are allowed

First C++ project so may be PEBCAK

In digging into this I noticed NormalType is set to PointXYZ here:
https://github.com/PointCloudLibrary/pcl/blame/5cb90708950a9dc40335c573645eab355c9b492d/gpu/features/include/pcl/gpu/features/features.hpp#L57

Trying to set it to PointNormal throws a new error on compile since the size of PointNormal doesn't match the size of the typedef here:
https://github.com/PointCloudLibrary/pcl/blob/master/gpu/features/src/internal.hpp#L59

Am I on the right track there? Is this worth pursuing or is this package going to be replaced by the code in the cuda dir?

Thanks!

[barnjamin]

Alright I got something that will give me curvature, not sure if its a hack or normal for C++

Is this an acceptable way to do this? Do you want something written up to document this?

Thanks,
Ben

...
  pcl::gpu::NormalEstimation ne;
  ne.setInputCloud(cloud_device);

  pcl::gpu::Feature::Normals normals_small;
  ne.setRadiusSearch(scale1, 100);
  ne.compute(normals_small);
  ...
  PointXYZ normals_small_host[cloud->points.size()];
  normals_small.download(&normals_small_host[0]);

  pcl::PointCloud<PointNormal>::Ptr small_normals(new pcl::PointCloud<PointNormal>);
  small_normals->width = cloud->points.size();
  small_normals->height = 1;
  for(int x=0; x<cloud->points.size(); x++){
        PointNormal np;

        Normal* n = (Normal*)&normals_small_host[x];
        PointXYZ p = cloud->points[x];

        np.x = p.x; np.y =p.y; np.z = p.z;
        np.normal_x = n->normal_x; np.normal_y = n->normal_y; np.normal_z = n->normal_z;
        np.curvature = n->curvature;
        small_normals->points.push_back(np);
  }

[SergioRAgostinho]

Hi Ben. Our gpu and cuda modules are not mature and the original authors are no longer active. I'm not gpu/cuda module pro, but your code looks fishy for the following reason: pcl::gpu::Feature::Normals are containers of PointXYZ which has the components x y and z, usually floats. Upon downloading to the host, you also populate a PointXYZ which has attributes x, y and z and usually span 16bytes (instead of the expected 12) for memory alignment reasons.
When you hardcore this cast

Normal* n = (Normal*)&normals_small_host[x];

the attributes normal_x, normal_y and normal_z will be filled properly, but n->curvature is completely memory trash. It's whatever is present in the next memory address of normals_small_host[x]. I would surprised if you got good values out of that attribute.

[barnjamin]

First, "harcore cast" is an awesome name.

Second, i _think_ it happens to work in this case. I'll take a closer look later but I got reasonable results in subsequent steps.

We are actually computing the curvature here
https://github.com/PointCloudLibrary/pcl/blob/master/gpu/features/src/normal_3d.cu#L162

It looks like space for 4 floats is reserved for the PointXYZ here unless I'm reading it wrong (entirely likely)
https://github.com/PointCloudLibrary/pcl/blob/master/common/include/pcl/impl/point_types.hpp#L278

[SergioRAgostinho]

We are actually computing the curvature here
https://github.com/PointCloudLibrary/pcl/blob/master/gpu/features/src/normal_3d.cu#L162

That's a good observation, your curvature is being stored in the w component. Now have a look at this
https://github.com/PointCloudLibrary/pcl/blob/d3d3d2964893be4c12d9020b26cda259cb04e4a2/common/include/pcl/impl/point_types.hpp#L179-L187

and this
https://github.com/PointCloudLibrary/pcl/blob/d3d3d2964893be4c12d9020b26cda259cb04e4a2/common/include/pcl/impl/point_types.hpp#L773-L785

These are the types you have on the host. pcl::PointXYZ type has a PCL_ADD_UNION_POINT4D inside and spans 16bytes but pcl::Normal actually spans 32 bytes (16 from the normal components and 16 for the curvature + padding). You're casting something with 32 bytes to something with 16 bytes. What your curvature components from the Normaltype is getting is the x component of the next PointXYZ.

This is why I mentioned "Hardcore Cast" :)

[barnjamin]

From pcd file:

...
-1.508478 -0.68691474 2.4120002 0.16903351 0.93719709 -0.30510524 **0.17954314**
-1.5167272 -0.68606037 2.4090002 **0.17954314** 0.93220568 -0.3142558 0.16487625
-1.5249511 -0.685206 2.4060001 0.16487625 0.9292767 -0.3305459 0.13595936
...

aaannnd you're right. last entry there is meant to be curvature which is the exact same value for the next row in normal_x.

So is there any way to get at the curvature being computed or is it left behind when I download from the gpu?

[SergioRAgostinho]

Fortunately i believe it should be there. I expect you'll find it at normals_small_host[x].data[3].

[taketwo]

data_c[3]? :) no, data[3] indeed.

[barnjamin]

like magic!

these look a lot less suspicious now.

...
0.35813245 -0.62076604 1.664 -0.10444337 0.66036856 -0.74364299 0.0048221406
0.35134944 -0.62113911 1.6650001 -0.095396467 0.64837533 -0.75532043 0.0044923765
0.34455803 -0.62151217 1.6660001 -0.087876782 0.64304596 -0.76076901 0.0044038729
0.33755559 -0.62151223 1.6660001 -0.084492736 0.63102406 -0.7711482 0.0038377375
...

For reference:

...
           np = normals_small_host[x];
            nps.normal_x = np.x; nps.normal_y = np.y; nps.normal_z = np.z;
            nps.curvature = np.data[3];
            small_normals->points.push_back(nps);
...

Thank you!

[SergioRAgostinho]

I'll close this one now. 👍