Vector: Expose musl/libc++/clang toolchain externally

[LucioFranco]

@a-rodin has built a pretty fantastic toolchain image that can compile rust binaries that include c++/c dependencies with cross-lang lto. This is something that is non-trivial and could benefit the wider rust community.

I suggest that we create a separate repo to keep track of this toolchain image and then have vectors specific builder add the dependencies like OpenSSL. This would also benefit us in the fact that we would not need to rebuild the whole image compiling llvm, rustc, and clang each time we would like to add a new dependency to the build/release process.

Reference https://github.com/timberio/vector/pull/945#issuecomment-536043589

[a-rodin]

Thanks @LucioFranco. I like this idea.

The cross-lang LTO was actually disabled later because typetag's dependency rust-ctor didn't support LLD. The problem seems to be solved recently in this PR, where, notably, there is a call from the maintainer for creating a test case using alternative linkers. I think we can publish a builder image with cross-lang LTO enabled and also contribute to rust-ctor by creating a test using our image for building.

[binarylogic]

This is a naive question, but would that actually benefit other Rust developers, or is it highly specific to Vector?

[binarylogic]

Please ignore my comment, I just read the conversation on the PR 😄

[a-rodin]

Additionally, I think we can unify it with the builder for armv7-unknown-linux-musleabihf.

[LucioFranco]

@a-rodin does clang support changing its target like rustc can? Meaning we can build multiple toolchains into the image?

[a-rodin]

Clang itself supports, although the libraries would need to be put into separate prefixes. However, I was thinking about having a single Dockerfile, but two images that are built with different build args, probably in CI.

[a-rodin]

Also, with multi-arch support in place we would be able to easily create builders for i386-unknown-linux-musl and aarch64-unknown-linux-musl.

[LucioFranco]

Yeah, that makes sense!

[a-rodin]

I think it makes sense to do this after #859, #1035, and #1036 are done (but not necessarily merged) to stabilize requirements for the exposed toolchains.

[a-rodin]

I've created a repository https://github.com/timberio/rust-toolchain. For now it contains only a Dockerfile which can be built with different args, the idea is to use supply different in CircleCI to create images for different target architectures.

[binarylogic]

Re-opening this since we want to move these images into the Vector repo and re-build the images each night (or maintain them in some way)

[binarylogic]

Noting, this should be part of our migration to GH actions.

[binarylogic]

Marking this as low priority since we seem to be moving towards Glibc builds.

[binarylogic]

Closing this since we're moving to glibc.