Pcl: [Ubuntu 19.10 CI] Out of memory (disk space)

[kunaltyagi]

Your Environment

• Operating System and version: CI ubuntu 19.04

Current Behavior

19.04 tests for out of core are failing due to low disk space

[kunaltyagi]

I think this is an issue with Azure but if not, we need more information. All PR are red and that's super confusing. Shall we disable 19.04 temporarily?

[taketwo]

How did we conclude that it's a OOM issue? Here is the relevant output from one of the last pipeline runs:

84: [----------] 12 tests from OutofcoreTest
84: [ RUN      ] OutofcoreTest.Outofcore_Constructors
84: [pcl::PCDWriter::writeBinaryCompressed] posix_fallocate errno: 2 strerror: No such file or directory
84: unknown file: Failure
84: C++ exception with description ": [pcl::PCDWriter::writeBinaryCompressed] Error during posix_fallocate ()!" thrown in the test body.
84: [  FAILED  ] OutofcoreTest.Outofcore_Constructors (10013 ms)
84: [ RUN      ] OutofcoreTest.Outofcore_ConstructorSafety
84: /__w/1/s/test/outofcore/test_outofcore.cpp:482: Failure
84: Value of: boost::filesystem::exists (filename_otreeB)
84:   Actual: false
84: Expected: true
84: No tree detected on disk. This test will fail. Perhaps this test was run out of order.

I identified the place where the debug print happens and the exception is thrown:

https://github.com/PointCloudLibrary/pcl/blob/2133857dd8ddd0e80602fc1602651a5afb10f934/io/include/pcl/io/impl/pcd_io.hpp#L373-L383

Interestingly, posix_fallocate man page says that it does not set _errno_. This implies that the error message that we print has nothing to do with the actual error. Also, this check in raw_fallocate does not make sense then:

https://github.com/PointCloudLibrary/pcl/blob/2133857dd8ddd0e80602fc1602651a5afb10f934/io/include/pcl/io/low_level_io.h#L175-L183

[kunaltyagi]

I based it on the fact that I got those errors (ENOENT -- No such file was found or the specified path name doesn't exist) when files can't be created to write on a constrained docker mount. I didn't think that it had something weird inside PCL code. My mistake.

[kunaltyagi]

I now know why I said OOM: Because I saw an error message saying so:

link to azure build

This seems to be separate issue from compilation failure. Screenshot:

[kunaltyagi]

Builds consistently red, still.

https://dev.azure.com/PointCloudLibrary/pcl/_build/results?buildId=12796&view=logs&j=4de25620-fe7f-583f-ecca-01dec01cc67e&t=471d7ae6-937d-5438-1f44-d8d5765977a5

Either due to this or #3696

[shrijitsingh99]

Currently, it seems as if the CI is failing in the build stage before even reaching testing.
Switching to CMAKE_BUILD_TYPE="Release" should keep the build size (less than 1G) smaller than CMAKE_BUILD_TYPE="RelWithDebInfo" which is over 10G.

Is there any issue in switching to the release build type as we don't need the debugging symbols in the CI pipeline do we?

[kunaltyagi]

First thing that comes to mind: The flags go from -O2 to -O3, increasing compile time for sure.

[SergioRAgostinho]

Is there any issue in switching to the release build type as we don't need the debugging symbols in the CI pipeline do we?

We definitely do not.

[shrijitsingh99]

First thing that comes to mind: The flags go from -O2 to -O3, increasing compile time for sure.

Yeah, there would be but an increase in time, but looking at the execution times doesn't seem on CI this would be that much of an increase.

Is there any issue in switching to the release build type as we don't need the debugging symbols in the CI pipeline do we?

We definitely do not.

Great! I have made a draft PR #3745. Hopefully, we can get the Ubuntu CI running green again.