mlpack build process halts the system

[mrityunjay-tripathi]

As I am new to mlpack, I might be unaware of certain things. But whenever I build (whether using the methods in the documentation or directly using vscode), it takes too much time and CPU resources. If I use all 4 cores, the system halts in 5-10 minutes and if only one or two cores are used it takes 1-1.5 hours to halt. Build process progresses without errors (upto 20-30%) but never completes. System specifications -

CPU - corei7 3rd gen 2.06GHZ
RAM - 8GB
Swap Memory - 2GB
Storage - 180GB SSD
OS - Ubuntu 18.04
Armadillo version - 8.400.0 (Entropy Bandit)

[himanshupathak21061998]

While using cmake you can turn off python binding and cli for faster build using flags like.

-DBUILD_CLI_EXECUTABLES=OFF 
-DBUILD_PYTHON_BINDINGS=OFF

and also can build a specific target using

make target_name

[kartikdutt18]

You can also turn off Julia bindings if needed, -D BUILD_JULIA_BINDINGS = OFF. I think using 2 cores with the aforementioned cmake options disabled should speed things up a bit. For reference you can see build time for Linux / Windows on azure pipelines from your PR.

[rcurtin]

It sounds like you're running out of RAM and your system is starting to swap. Try also building on only one core and that should help. I'm quite surprised 8GB is having problems---are there perhaps other processes running on your system that are causing much of your RAM to be unavailable? mlpack typically will use ~1.5GB per core when building (or somewhere around that; I haven't exactly profiled it).

[mrityunjay-tripathi]

Thank you, @rcurtin @kartikdutt18 @himanshupathak21061998 for your useful suggestions. Using only one core helps (to not let the RAM overflow). Also turning off the python and Julia bindings increase speed and the build process completes within a few minutes.
But anyway using all cores makes the RAM overflow (no other programs open except System Monitor) even if I turnoff Python and Julia bindings.

[kartikdutt18]

Generally on azure pipelines, they build mlpack with 2 cores, try that it should work fine.

[mrityunjay-tripathi]

Ok. Let me try that also.

PS:
Yes! it's the best configuration and a max of 5GB RAM is used during the build process (system processes + build process). I think the documentation should explicitly mention this.