Apollo: Show Pointcloud error Dreamview

[krishnatoshniwal]

While toggling the Point Cloud button in Dreamview in the Layer Menu, I was running into an issue. I confirmed that Pose information and Point clouds are accessible (#6909). However, the moment I turn on the Point Cloud toggle, Dreamview stops working and displays this error in Console : Connection to the server has been lost. I checked the dreamview log file but found no mention of any errors. I wanted to confirm if the show Point Cloud function works for Apollo 3.5 and if yes, can the error be resolved?

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 14.04): Ubuntu 18.04
• Apollo installed from (source or binary): source
• Apollo version (1.0, 1.5, 2.0, 2.5, 3.0): 3.5

[luqiang21]

@krishnatoshniwal It is not working for us. When we enabled Point Cloud, dreamview crashed.

[siyangy]

Can you take a look at the coredump file generated upon crash? Should be able to see the stacktrace from there.

[krishnatoshniwal]

I got the following output:

Reading symbols from bazel-bin/modules/dreamview/dreamview...done.
[New LWP 16733]
[New LWP 16731]
[New LWP 16737]
[New LWP 16740]
[New LWP 16716]
[New LWP 16750]
[New LWP 16729]
[New LWP 16730]
[New LWP 16745]
[New LWP 16757]
[New LWP 16768]
[New LWP 16743]
[New LWP 16756]
[New LWP 16742]
[New LWP 16744]
[New LWP 16759]
[New LWP 16746]
[New LWP 16738]
[New LWP 16771]
[New LWP 16735]
[New LWP 16790]
[New LWP 16779]
[New LWP 16747]
[New LWP 16748]
[New LWP 16783]
[New LWP 16772]
[New LWP 16796]
[New LWP 16741]
[New LWP 16763]
[New LWP 16793]
[New LWP 16749]
[New LWP 16751]
[New LWP 16773]
[New LWP 16791]
[New LWP 16739]
[New LWP 16798]
[New LWP 16753]
[New LWP 16752]
[New LWP 16774]
[New LWP 16792]
[New LWP 16732]
[New LWP 16758]
[New LWP 16778]
[New LWP 16777]
[New LWP 16760]
[New LWP 16780]
[New LWP 16761]
[New LWP 16782]
[New LWP 16762]
[New LWP 16785]
[New LWP 16765]
[New LWP 16789]
[New LWP 16767]
[New LWP 16769]
[New LWP 16775]
[New LWP 16776]
[New LWP 16784]
[New LWP 16734]
[New LWP 16736]
[New LWP 16754]
[New LWP 16755]
[New LWP 16764]
[New LWP 16770]
[New LWP 16781]
[New LWP 16794]
[New LWP 16795]
[New LWP 16797]
[New LWP 16766]
[New LWP 16786]
[New LWP 16787]
[New LWP 16788]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/apollo/bazel-bin/modules/dreamview/dreamview --flagfile=/apollo/modules/common'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f2b267c5c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56

[siyangy]

Please run a bt to get the actual stacktrace.

[krishnatoshniwal]

Apologies! I believe this is the backtrace.

#0  0x00007f8e96f5dc37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f8e96f61028 in __GI_abort () at abort.c:89
#2  0x00007f8e97566535 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007f8e975646d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007f8e97564703 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f8e97564922 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x0000000000fcabdd in google::protobuf::internal::LogMessage::Finish (this=0x7f8d2effeec0) at external/com_google_protobuf/src/google/protobuf/stubs/common.cc:268
#7  0x0000000000fcac18 in google::protobuf::internal::LogFinisher::operator= (this=0x7f8d2effee9f, other=...) at external/com_google_protobuf/src/google/protobuf/stubs/common.cc:276
#8  0x000000000068becf in google::protobuf::internal::RepeatedPtrFieldBase::Get<google::protobuf::RepeatedPtrField<apollo::drivers::PointXYZIT>::TypeHandler> (this=0x7f8d0c1b3c30, 
    index=10971) at external/com_google_protobuf/src/google/protobuf/repeated_field.h:1482
#9  0x000000000068922e in google::protobuf::RepeatedPtrField<apollo::drivers::PointXYZIT>::Get (this=0x7f8d0c1b3c30, index=10971)
    at external/com_google_protobuf/src/google/protobuf/repeated_field.h:1919
#10 0x000000000068737c in apollo::drivers::PointCloud::point (this=0x7f8d0c1b3c18, index=10971) at bazel-out/local-dbg/genfiles/modules/drivers/proto/pointcloud.pb.h:637
#11 0x000000000068538b in apollo::dreamview::PointCloudUpdater::UpdatePointCloud (this=0x3cbdc50, point_cloud=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18)
    at modules/dreamview/backend/point_cloud/point_cloud_updater.cc:166
#12 0x0000000000684fec in apollo::dreamview::PointCloudUpdater::__lambda66::operator() (__closure=0x3d46a60, msg=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18)
    at modules/dreamview/backend/point_cloud/point_cloud_updater.cc:129
#13 0x0000000000686314 in std::_Function_handler<void(const std::shared_ptr<apollo::drivers::PointCloud>&), apollo::dreamview::PointCloudUpdater::Start()::__lambda66>::_M_invoke(const std::_Any_data &, const std::shared_ptr<apollo::drivers::PointCloud> &) (__functor=..., __args#0=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18)
    at /usr/include/c++/4.8/functional:2071
#14 0x00000000006981eb in std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)>::operator()(std::shared_ptr<apollo::drivers::PointCloud> const&) const (
    this=0x3d3b028, __args#0=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18) at /usr/include/c++/4.8/functional:2471
#15 0x0000000000694bd5 in apollo::cyber::Reader<apollo::drivers::PointCloud>::Init()::{lambda(std::shared_ptr<apollo::drivers::PointCloud> const&)#1}::operator()(std::shared_ptr<apollo::drivers::PointCloud> const&) const (__closure=0x3d3f620, msg=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18) at ./cyber/node/reader.h:142
#16 0x000000000069fd2f in std::_Function_handler<void (std::shared_ptr<apollo::drivers::PointCloud> const&), apollo::cyber::Reader<apollo::drivers::PointCloud>::Init()::{lambda(std::shared_ptr<apollo::drivers::PointCloud> const&)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptr<apollo::drivers::PointCloud> const&) (__functor=..., 
    __args#0=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18) at /usr/include/c++/4.8/functional:2071
#17 0x00000000006981eb in std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)>::operator()(std::shared_ptr<apollo::drivers::PointCloud> const&) const (
    this=0x3d3f6e0, __args#0=std::shared_ptr (count 3, weak 0) 0x7f8d0c1b3c18) at /usr/include/c++/4.8/functional:2471
#18 0x00000000006984b0 in apollo::cyber::croutine::RoutineFactory apollo::cyber::croutine::CreateRoutineFactory<apollo::drivers::PointCloud, std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)> >(std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)>&&, std::shared_ptr<apollo::cyber::data::DataVisitor<apollo::drivers::PointCloud, apollo::cyber::NullType, apollo::cyber::NullType, apollo::cyber::NullType> > const&)::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (__closure=0x3d3f6d0)
    at ./cyber/croutine/routine_factory.h:61
#19 0x00000000006a9ea2 in std::_Function_handler<void (), apollo::cyber::croutine::RoutineFactory apollo::cyber::croutine::CreateRoutineFactory<apollo::drivers::PointCloud, std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)> >(std::function<void (std::shared_ptr<apollo::drivers::PointCloud> const&)>&&, std::shared_ptr<apollo::cyber::data::DataVisitor<apollo::drivers::PointCloud, apollo::cyber::NullType, apollo::cyber::NullType, apollo::cyber::NullType> > const&)::{lambda()#1}::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/4.8/functional:2071
#20 0x00000000004ad786 in std::function<void ()>::operator()() const (this=0x3d3f670) at /usr/include/c++/4.8/functional:2471
#21 0x0000000000d400c0 in apollo::cyber::croutine::CRoutine::Run (this=0x3d3f658) at ./cyber/croutine/croutine.h:143
#22 0x0000000000d3f715 in apollo::cyber::croutine::(anonymous namespace)::CRoutineEntry (arg=0x3d3f658) at cyber/croutine/croutine.cc:43
#23 0x0000000000000000 in ?? ()

[siyangy]

If you take a look at the code from https://github.com/ApolloAuto/apollo/blob/master/modules/dreamview/backend/point_cloud/point_cloud_updater.cc#L154, it's trying to copy the pointcloud cloud data from a protobuf data payload into pcl data structure. The particular error you ran into was due to the size of the point data doesn't match with its width*height. Please take a look at your original pointcloud data and make sure it's not corrupted with violated invariance assumption.

[krishnatoshniwal]

Thank you! I fixed the issue with the pointcloud data and dreamview doesn't crash. However I still am unable to see the pointcloud in Dreamview. In Dreamview.INFO I see the message that the point cloud is getting filtered with the message being : "filtered point cloud data size: XXXX". Should I be seeing any other info messages related to the pointcloud and also if there's a way to ensure that I see the pointcloud being rendered on Dreamview. Thanks again!

[siyangy]

If you see that log, it probably indicates that the pointcloud data has been sent to your frontend. Feel free to dig further around the data itself. I imagine incorrect coordinate values could cause this.

[lucianzhong]

@siyangy

Could you please explain more about the error "The particular error you ran into was due to the size of the point data doesn't match with its width*height."

Which line of the backtrace shows the error?

Thanks a lot.