Pcl: Crash when calling updatePointCloud

[tmaslach]

My Environment:
Operating System: Windows 10
Compiler VS2019
PCL Version: 1.9.1
VTK Version: 8.2.0

Context
(Issue has been confirmed by someone else with same VTK/PCL versions, but with Ubuntu 18.04 - not sure about compiler that was used)

I originally posted this issue on the VTK forums, as I incorrectly assumed it was a VTK issue: https://gitlab.kitware.com/vtk/vtk/issues/17688. You can look there for more background or simply continue on here.

In my attached example, I am adding a point cloud and then continuously updating that point cloud. The size of the point cloud varies (which was mentioned as a possible reason for triggering the crash in the VTK ticket).

The crash happens in VTK's vtkOpenGLIndexBufferObject::AppendPointIndexBuffer. This code is the most direct culprit:

  size_t targetSize = indexArray.size() +
    cells->GetNumberOfConnectivityEntries() -
    cells->GetNumberOfCells();

NumberOfCells is greater than NumberOfConnectivityEntries. indexArray is of size 0. This leads to a negative value. Since size_t is unsigned, we end up with an insanely large number. Reserving a vector of that size (which happens right after) causes the crash.

If I remove the point cloud and add again, I avoid getting the crash. This is a suitable workaround, but I think this shows there is a problem (most likely) in updatePointCloud.

Code to Reproduce
I am attaching an example that can be used to reproduce this on windows 10. I suspect it will happen on other platforms as well, but did not confirm.

There is an #if on line 45. If set to 1/true, it will use removePointCloud/addPointCloud instead, which works.

Here is a project to recreate the issue:
PointCloudCrashPCL.zip

Possible Solution
I'm not very familiar with VTK, so I can't help here. But, Adrian on the VTK ticket suspects the NumberOfCells is not being updated when calling updatePointCloud. Please read his comment on the ticket I provided above.

[raahilsha-z]

Thank you so much for investigating this! I've been having the same issue for about 3 months, and first noticed it when I updated my vcpkg and started using VTK 8.2 instead of 8.1 on Windows 10. Since I'm using vcpkg, I haven't been able to try PCL 1.9.1 with VTK 8.1.

I've also confirmed that this issue happens regardless of what camera is being used as input (tested with PCL's OpenNI2 grabber, as well as my own custom Azure Kinect, Kinect v1, Realsense 2, and Structure Core grabbers).

@tmaslach 's example uses PointXYZI, and I can confirm that the crash also happens with PointXYZ, XYZRGBA, and XYZRGB.

[kunaltyagi]

It turns out that there may be a bug in the updatePointCloud() method of pcl::visualization::PCLVisualizer. Namely, GetNumberOfCells() appears to not get updated. If you have a large point cloud and then you update the visualizer with a smaller point cloud using the same id, then GetNumberOfCells() may be larger than GetNumberOfConnectivityEntries() which should actually never happen if the point cloud is consistent.
My workaround, which also supports my observation, is to call removePointCloud() followed by addPointCloud() instead of updatePointCloud().

Reproducing the hack/solution here for ease of discussion. Thanks to Adrian (@aleu)

[SiddhantNadkarni]

I'm facing the same issue, if there is a significant decrease in the point cloud size while using updatePointCloud() method to refresh viewer, the visualizer crashes. Has anyone managed to resolve the issue apart from the hack/workaround mentioned by Kunal?

Thanks!

[kunaltyagi]

If anyone is interested, they can start a PR to refactor the updatePointCloud.

The main difference is that calling add and remove calls other functions to update the cloud, while calling update performs those operations inside the update call (and there might be a bug in those lines)

[sglimberg]

Just want to bring some more awareness to this issue. We are experiencing the exact same issue after updating PCL and VTK versions (pcl-1.10.0, vtk-8.2, Windows 10).

I did the workaround with remove and add, but I assume this will cause unnecessary overhead every time?

[SiddhantNadkarni]

Yes - the refresh rate using add and remove is lower than that with updatePointCloud() method.

[jafrado]

As an optimization, you can check if the new cloud has more points then the previous cloud, if so you can safely call updatePointCloud(), else remove and addPointCloud. If your lasers take time to ramp, you'll need to do this. I also found this issue a while back and did the above workaround. It does reduce performance, slightly but it is solid and has passed many tests.
Environment: Windows 10, 1.91 PCL, VTK 8.2.0, vcpkg builds.

[SiddhantNadkarni]

Thanks a lot for the suggestions @jafrado , will try them.

[kunaltyagi]

you can check if the new cloud has more points then the previous cloud

Thanks @jafrado, this is a nice insight into what to look for in the bug

[sglimberg]

I did a quick performance test on my laptop (Intel i7 2.6GHz) with point cloud sizes of approximately 80,000 points.

The updatePointCloud() takes on average 5.3ms
The removePointCloud and addPointCloud() takes on average 6.7

So there is a a penalty here, about 25% increase. Hope that a fix is coming up.

[jafrado]

Yes, we should fix this. I will do some more testing this week and can commit to a stack trace or two, I have way too many projects on my hands, if someone can fix at this, I will buy them beer for a week :-)

[MaxFleur]

I'm so glad I found this page. I have the same issue, but interesting is that I have it with a very small set of points.
So I was right, the updatePointCloud seems to have this issue... hope this will be fixed soon. Calling remove and then add again works for now.

[taketwo]

Everyone who reported the problem: we have a PR that supposedly fixes the problem, please give it a try.

[jafrado]

Cool! Looks like I owe Larsg some beers!

On Sun, May 3, 2020 at 10:57 AM Sergey Alexandrov [e-mail]
wrote:

Everyone who reported the problem: we have a PR that supposedly fixes the
problem, please give it a try.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PointCloudLibrary/pcl/issues/3452#issuecomment-623152871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXSUDE6VRTSBZNSO3L2B4TRPWV7HANCNFSM4JJ3CAHA
.

[jafrado]

Perfect! Works great for me. Thank you Lars!

On Sun, May 3, 2020 at 3:06 PM James Dougherty [e-mail] wrote:

Cool! Looks like I owe Larsg some beers!

On Sun, May 3, 2020 at 10:57 AM Sergey Alexandrov <
[email protected]> wrote:

Everyone who reported the problem: we have a PR that supposedly fixes the
problem, please give it a try.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PointCloudLibrary/pcl/issues/3452#issuecomment-623152871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXSUDE6VRTSBZNSO3L2B4TRPWV7HANCNFSM4JJ3CAHA
.

[jafrado]

&& yes, I sent him a gift :-)

On Sun, May 3, 2020 at 3:23 PM James Dougherty [e-mail] wrote:

Perfect! Works great for me. Thank you Lars!

On Sun, May 3, 2020 at 3:06 PM James Dougherty [e-mail] wrote:

Cool! Looks like I owe Larsg some beers!

On Sun, May 3, 2020 at 10:57 AM Sergey Alexandrov <
[email protected]> wrote:

Everyone who reported the problem: we have a PR that supposedly fixes
the problem, please give it a try.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PointCloudLibrary/pcl/issues/3452#issuecomment-623152871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXSUDE6VRTSBZNSO3L2B4TRPWV7HANCNFSM4JJ3CAHA
.

[larshg]

Thanks for the gift - if you ever visit Denmark, I'll buy a beer :)

[jafrado]

my pleasure; enjoy!