Falco: Build improvements: Dynamic builds

[fntlnz]

What would you like to be added:

At the moment, the main Falco CMakeLists.txt and its children have hard-coded compilation flags to always compile falco as a static binary.

While has its advantages (no extra dependencies), using static binaries as a packaging medium can be cause of any sort of issues, like:

• Big final binary size
• Difficulties in statically building/linking all the dependencies in various operating systems/kernels/glibc versions
• We don't rely on distributions to take care of patches, and most importantly, security patches in our dependencies (because we don't use OS packages for them)
• The previous point is particularly important, since we have a webserver and a gRPC server. Both need an SSL implementation. We are statically linking OpenSSL to a version that is > 1year old
• It's difficult to keep dependencies updated in general, we would rather declare the versions we work with
• Building is extremely slow and this is BAD contributor experience

Of course, one might say: "Yes but packaging becomes more difficult"

I would argue with that, we have three main distribution mediums:

• rpm packge
• deb package
• container image (aka. docker image)

For the rpm and deb package we can easily declare their dependencies in them instead.

For the container image we can install the needed dependencies directly in the container.

This does not affect the kernel module and the bpf probe. FTR I'm trying to refactor the bpf probe in a "compile once and run everywhere" fashion in issue #942 .

Why is this needed:

Because we all deserve better builds.

[fntlnz]

Here's the action plan in form of the dependencies we have.

Dates in the dd/mm/yyyy format.

| name | current used version | latest available version | current used release date | latest available release date | project status | ubuntu packages | fedora packages | centos packages | debian packages | arch linux packages | alpine packages | alpine edge packages |
|---------------|----------------------|--------------------------|---------------------------|-------------------------------|----------------|----------------------------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------|-------------------------------------------------|----------------------|--------------------------------------------------------|--------------------------------------------------------|
| libscap | falco/0.18.0 | falco/0.18.0 | 31/10/2018 | 31/10/2018 | maintained | n/a | n/a | n/a | n/a | n/a | n/a | |
| libsinsp | falco/0.18.0 | falco/0.18.0 | 31/10/2018 | 31/10/2018 | maintained | n/a | n/a | n/a | n/a | n/a | n/a | |
| zlib | 1.2.11 | 1.2.11 | 15/01/2017 | 15/01/2017 | maintained | zlib1g-dev=1:1.2.11.dfsg-0ubuntu2, zlib1g-dev=1:1.2.11.dfsg-0ubuntu2 | zlib=1.2.11-19.fc30, zlib-devel=1.2.11-19.fc30 | zlib=1.2.11-10.el8, zlib-devel=1.2.11-10.el8 | zlib1g=1:1.2.11.dfsg-1 | zlib=1:1.2.11-4 | zlib-dev-1.2.11-r1, zlib-1.2.11-r1 | zlib-dev-1.2.11-r3, zlib-1.2.11-r3 |
| jq | 1.5 | 1.6 | 16/08/2015 | 02/11/2018 | maintained | jq=1.5+dfsg-2, libjq-dev=1.5+dfsg-2 | jq=1.6-2.fc30 | jq=1.5-12.el8 | jq=1.5+dfsg-2+b1 | jq=1.6-2 | jq-dev-1.6-r0, jq-1.6-r0 | jq-dev-1.6-r0, jq-1.6-r0 |
| nlohmann-json | 3.3.0 | 3.7.3 | 05/10/2018 | 17/11/2019 | maintained | n/a | n/a | n/a | n/a | n/a | n/a | |
| ncurses | 6.0 | 6.1 | 08/08/2015 | 27/01/2018 | maintained | libncurses5-dev=6.1-1ubuntu1.18.04, libncurses5=6.1-1ubuntu1.18.04 | ncurses=6.1-10.20180923.fc30 | ncurses=6.1-7.20180224.el8, ncurses-devel=6.1-7.20180224.el8 | libncurses-dev=6.1+20181013-2+deb10u2 | ncurses=6.1-7 | ncurses-dev-6.1_p20190518-r0, ncurses-6.1_p20190518-r0 | ncurses-dev-6.1_p20191130-r0, ncurses-6.1_p20191130-r0 |
| libb64 | 1.2 | 1.2 | 28/06/2013 | 28/06/2013 | unmaintained | libb64-0d=1.2-4, libb64-dev=1.2-4 | libb64-devel=1.2-4.fc30 | n/a | libb64-0d=1.2-5,libb64-dev=1.2-5 | libb64=1.2.1-3 | n/a | n/a |
| yamlcpp | 0.6.2 | 0.6.3 | 06/03/2018 | 25/09/2019 | maintained | libyaml-cpp-dev=0.5.2-4ubuntu1 | yaml-cpp=0.6.2-1.fc30, yaml-cpp-devel=0.6.2-1.fc30 | n/a | libyaml-cpp-dev=0.6.2-4, libyaml-cpp0.6=0.6.2-4 | yaml-cpp=0.6.3-1 | yaml-cpp-0.6.2-r2, yaml-cpp-dev-0.6.2-r2 | yaml-cpp-0.6.3-r0 yaml-cpp-dev-0.6.3-r0 |
| OpenSSL | 1.0.2n | 1.1.1c | 27/03/2018 | 10/09/2019 | maintained | openssl=1.1.1-1ubuntu2.1~18.04.5 | openssl=1:1.1.1d-2.fc30, openssl-devel=1:1.1.1d-2.fc30 | openssl=1:1.1.1-8.el8, openssl-devel=1:1.1.1-8.el8 | openssl=1.1.1d-0+deb10u2 | openssl=1.1.1.d-2 | openssl-1.1.1d-r0 | openssl-1.1.1d-r2 |
| libcurl | 7.61.0 | 7.67.0 | 16/07/2018 | 06/11/2019 | maintained | curl=7.58.0-2ubuntu3.8 | libcurl=7.65.3-4.fc30, libcurl-devel=7.65.3-4.fc30 | libcurl=7.61.1-8.el8, libcurl-devel=7.61.1-8.el8 | libcurl4=7.64.0-4 | curl=7.67.0-3 | libcurl-7.66.0-r0, curl-dev-7.66.0-r0 | libcurl-7.67.0-r0, curl-dev-7.67.0-r0 |
| LuaJIT | 2.0.3 | 2.0.5 | 19/02/2013 | 01/05/2017 | maintained | luajit=2.1.0~beta3+dfsg-5.1 | luajit=2.1.0-0.9beta3.fc30, luajit-devel=2.1.0-0.9beta3.fc30 | n/a | luajit=2.1.0~beta3+dfsg-5.1 | luajit=2.0.5-2 | luajit-2.1.0_beta3-r4, luajit-dev-2.1.0_beta3-r4 | luajit-2.1.0_beta3-r5, luajit-dev-2.1.0_beta3-r5 |
| lpeg | 1.0 | 1.2 | nd | nd | nd | lua-lpeg-dev=1.0.0-2ubuntu0.18.04.1, lua-lpeg=1.0.0-2ubuntu0.18.04.1 | lua-lpeg=1.0.1-10.fc30 | lua-lpeg=1.0.1-6.el8 | lua-lpeg-dev=1.0.0-2, lua-lpeg=1.0.0-2 | lua-lpeg=1.0.2-1 | lua-lpeg-dev-1.0.1-r4, lua-lpeg-1.0.1-r4 | lua-lpeg-dev-1.0.1-r4, lua-lpeg-1.0.1-r4 |
| Libyaml | 0.1.4 | 0.2.2 | 24/12/2012 | 13/03/2019 | maintained | libyaml-dev=0.1.7-2ubuntu3 | libyaml=0.2.1-5.fc30, libyaml-devel=0.2.1-5.fc30 | libyaml=0.1.7-5.el8 | libyaml-dev=0.2.1-1, libyaml-0-2=0.2.1-1 | libyaml=0.2.2-1 | yaml-0.2.2-r1 | yaml-0.2.2-r1 |
| lyaml | 6.0 | 6.2.4 | 27/07/2015 | 21/07/2019 | maintained | n/a | n/a | n/a | n/a | n/a | lua-lyaml-6.2-r3 | lua-lyaml-6.2-r3 |
| civetweb | 1.11 | 1.11 | 10/09/2018 | 10/09/2018 | maintained | n/a | n/a | n/a | n/a | n/a | | |
| cares | 1.13.0 | 1.15.0 | 20/06/2017 | 23/10/2018 | maintained | libc-ares2=1.14.0-1, libc-ares-dev=1.14.0-1 | c-ares=1.15.0-3.fc30, c-ares-devel=1.15.0-3.fc30 | c-ares=1.13.0-5.el8, c-ares-devel=1.13.0-5.el8 | libc-ares2=1.14.0-1,libc-ares-dev=1.14.0-1 | c-ares=1.15.0-1 | c-ares-dev-1.15.0-r0, c-ares-1.15.0-r0 | c-ares-1.15.0-r0 c-ares-dev-1.15.0-r0 |
| protobuf | 3.8.0 | 3.11.1 | 29/05/2019 | 03/12/2019 | maintained | libprotobuf-dev=3.0.0-9.1ubuntu1 | protobuf=3.6.1-3.fc30 | protobuf=3.5.0-7.el8 | libprotobuf-dev=3.6.1.3-2 | protobuf=3.10.1-1 | protobuf-3.6.1-r1 | protobuf-c-1.3.2-r3, protobuf-dev-3.11.1-r0 |
| tbb | 2018_U5 | 2019_U9 | 19/06/2018 | 10/10/2019 | maintained | libtbb-dev=2017~U7-8, libtbb2=2017~U7-8 | tbb-devel=2019.8-3.fc30, tbb=2019.8-3.fc30 | tbb=2018.2-9.el8, tbb-dev=2018.2-9.el8 | libtbb-dev=2018~U6-4, libtbb2=2018~U6-4 | libbtbb=2018.12.R1-1 | n/a | n/a |
| gRPC | 1.25.0 | 1.25.0 | 06/11/2019 | 06/11/2019 | maintained | libgrpc-dev=1.3.2-1.1~build1, libgrpc++1=1.3.2-1.1~build1 | grpc=1.18.0-2.fc30, grpc-devel=1.18.0-2.fc30 | n/a | libgrpc-dev=1.16.1-1, libgrpc6=1.16.1-1 | grpc=1.25.0-3 | n/a | grpc-dev-1.25.0-r1, grpc-1.25.0-r1 |

Appendix

1. I also added release dates to emphasize the fact that having the OS dealing with dependencies and dependencies dependencies is a better idea than doing it ourselves, we even have dependencies as old as 2012.

1. libscap and libsinsp will still be compiled into Falco but their dependencies will be not. As an alternative in future we can look into installing the Sysdig package as a whole but doesn't seem convenient from a user point of view because they will then have all the other sysdig tooling while they only wanted Falco. A better alternative would be to split libsinsp and libscap from the Sysdig repository and let them have their own packages.

2. IMPORTANT: It's important to note that OpenSSL has different release series, the LTS we use now, 1.0.2 is EOL on Dec 31th 2019 - read here - this is why in the list I went with the 1.1.1c as latest available.

3. Talking with @leodido - we acknowledge that when we are ready with the Inputs API (can replace civetweb with an HTTP external Input), we do a gRPC output Webhook, and the new rules engine we can drop: civetweb, lyaml, libyaml, lpeg, LuaJIT, cURL.

4. Not sure if we need b64 still, and we can send a patch to libscap to avoid ncurses in builds that don't use that feature

5. for nlohmann-json we will keep building it since it's an header-only dependency

6. I already see our builds going from 40 minutes to 1

[fntlnz]

For future documentation purposes, to build, Falco also requires the openssl binary to be available in the system when using the new dynamic configuration.

[kris-nova]

We might want to check out the AUR as well once this is simplified

[fntlnz]

Another dependency we need to have is the cpack binary whenever the user want to create deb and rpm packages. Along with rpmbuild for rpm packages.