Virtual-environments: Add C++ support (GCC and Clang) on all platforms

[kcgen]

Tool information

• Tool names: defacto C++ compilers: GCC and Clang
• Add or update? Add
• Desired versions: Latest version available in standard package repositories. Currently these are:

| Compiler | Version | OS | Package Manager(s) |
|---|---|---|---|
| GCC | 9 | all | apt, Brew, MacPorts, and MSYS2 |
| Clang | 8 | Ubuntu-19.04 | apt |
| Clang | 9 | macOS and Windows | Brew and Chocolatey |

• Approximate size: Clang depends on GCC (that's why it often makes sense to simply install both; in addition to their technical differences). Typically the combo is a couple GB on Windows and smaller on macOS and Linux.

• Brief description of packages: These C++ compilers are used to build literally every C++ autotools/cmake project out there (tens of thousands of open-source applications and tools; perhaps more on GitHub).

  The main goal of this request is to allow the vast majority of open source C/C++ packages to be ./configure'd and built with either compiler (assuming the user pulls in any package-specific dependencies) "out of the box".

• Homepages: GCC - https://gcc.gnu.org/, Clang - https://clang.llvm.org/

Virtual environments affected

• :ballot_box_with_check: macOS 10.15
• :ballot_box_with_check:Ubuntu 16.04 LTS
• :ballot_box_with_check: Ubuntu 18.04 LTS
• :ballot_box_with_check: Windows Server 2016 R2
• :ballot_box_with_check: Windows Server 2019

Can they be installed during the build?
Yes. I'm currently installing the following in my workflow:

| Compilers | OS | Package Manager | Time |
|---|---|---|---|
| GCC and Clang |Ubuntu | apt | < 60s |
| GCC | macOS | Brew and MacPorts | between 3 min and 5 min |
| GCC and Clang | Windows | MSYS2's pacman | between 8 min and 12 min |

Compiling my application takes less time than installing these.

Because of this excessive time and resource usage, I made use of GitHub's new cache feature available in CI workflows, however, the cache limit is (currently) too small to save most of these development stacks.

Because it seems absurd that GitHub's cache size cannot accommodate C++ "hello world", I flagged
it here to which GitHub staff member @chrispat suggested that I shouldn't be caching C++ dependencies and instead they should be available in the baseline VMs.

Are you willing to submit a PR?
Yes

[stevencpp]

I would prefer to have clang-9 installed. Most people should be using the latest release. We could also have both the default clang (version 8) and clang-9 installed.

[kcgen]

@stevencpp , yes, I would prefer that the latest compiler (as determined by the package manager), be installed by default. Can you let me know on which OS clang-9 is available via the package manager? Thank you; I will update my issue.

[stevencpp]

On Ubuntu I use this to install it as clang-9 (takes about a minute):
```
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key 2>/dev/null | sudo apt-key add -
sudo add-apt-repository 'deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main' -y
sudo apt-get update -q
sudo apt-get install -y clang-9 lld-9 libc++-9-dev libc++abi-9-dev clang-tools-9

and on windows I use this (takes 25 seconds):

curl -fsSL -o LLVM9.exe https://releases.llvm.org/9.0.0/LLVM-9.0.0-win64.exe
7z x LLVM9.exe -y -o"C:/Program Files/LLVM"
```
I haven't tried setting it up on macOS yet.

[kcgen]

Interesting; even though Clang-9 is not in Ubuntu's 19.04 baseline APT repo, nor in macOS's Brew or MacPorts repos, nor in MSYS2's pacman repo -- I do see that these are official releases from LLVM.org.

I would prefer to use latest version per the repos due to the level of integration they provide (ie: pacman under msys2 will pull in dependencies and allow most autotools packages to build - where as a standalone exe installer might be totally disconnected from the msys2 environment). I realize the trade-off here is having to wait a bit longer for the lastest version to be supported by the various operating system repositories.

The main goal of this request is to allow the vast majority of open source C/C++ packages to be ./configure'd and built with either compiler (assuming the user pulls in any package-specific dependencies), as well as support the creation or modifications of similar autotools/cmake type C++ projects "out of the box".

I've amended the version line in the request with a mix of this goal combined with your suggestion; thanks @stevencpp!

[stevencpp]

Chocolatey https://chocolatey.org/packages/llvm and scoop https://github.com/ScoopInstaller/Main/blob/master/bucket/llvm.json have 9.0.0.
https://packages.msys2.org/base/mingw-w64-clang says pacman should have 9.0.0 as well, but I'd rather not use msys for clang. GCC may need msys/cygwin, but for clang I'm guessing most people use it without msys on windows (especially now that Visual Studio comes with clang-cl support built in). The official binary package includes the whole toolchain so it doesn't depend on GCC and cmake works fine with it, no need to depend on msys.
Also https://formulae.brew.sh/formula/llvm suggests brew on macOS should have 9 by default as well ?

[kcgen]

@stevencpp, you've got more experience with Clang under windows than myself; thanks for all of this! I've updated the version block.

I left out mentioning that Scoop or MSYS2 could be used to install Clang 9 because it seems like Chocolatey is the prefered approach (want to keep this to the "best" recommended installations only). Does it read OK now?

[stevencpp]

Seems alright.

There's a proposed llvm-toolchain-9 package for apt https://launchpad.net/ubuntu/+source/llvm-toolchain-9/1:9-2~ubuntu18.04.1 so that might land as well soon ?

[zbeekman]

If you're going to install the latest stable GCC it would be really nice to include GFortran along with C/C++. On some package managers (Homebrew) you get gfortran and g++ with gcc (e.g., brew install gcc) but on others (at least linux) you need to apt-get install gfortran-<version>, possibly after adding the toolchain-r/test ppa.

[ivan-nosar]

Good day!
We have added gcc, gfortran and clang tools on Windows and Ubuntu virtual environments in #369

[kcgen]

Good day!
We have added gcc, gfortran and clang tools on Windows and Ubuntu virtual environments in #369

Thank you @ivan-nosar ! This will dramatically reduce our workflow times.

[al-cheb]

Hello, All

We have added GCC and Clang for Ubuntu in #369 and MacOs. We are going to add Windows support in scope of MSYS2 feature, after https://github.com/actions/virtual-environments/pull/355 PR will be merged. It should start rolling out early next week, once the rollout start it shows up as a pre-release here: https://github.com/actions/virtual-environments/releases

After that it takes about 6-7 days to reach all accounts (assuming no delays).

[kcgen]

@al-cheb , excellent! Will monitor the release status and am looking forward to putting this to use.

[miketimofeev]

@krcroft FYI ubuntu and macOS images have been rolled out.

[kcgen]

@krcroft FYI ubuntu and macOS images have been rolled out.

Confirmed in the latest CI workflows.

macOS:

2020-02-20T20:42:54.7165320Z Warning: gcc 9.2.0_3 is already installed and up-to-date

Ubuntu:

2020-02-20T20:42:42.1321351Z clang-8 is already the newest version

2020-02-20T20:42:40.7763100Z build-essential is already the newest version (12.4ubuntu1).
2020-02-20T20:42:40.7768160Z libtool is already the newest version (2.4.6-2).
2020-02-20T20:42:40.7768382Z libtool set to manually installed.
2020-02-20T20:42:40.7770189Z g++-9 is already the newest version (9.2.1-17ubuntu1~18.04.1)

Our package install phases have dropped from mutliple minutes to sub-60s for macOS and roughly 15s in the case of Linux! Huge win for our (personal) time and GitHub's power usage :-). Thank you @miketimofeev .

[maxim-lobanov]

Reoped since GCC and CLang were not added to Windows platform yet.

[kcgen]

Reoped since GCC and CLang were not added to Windows platform yet.

These are the most important when it comes to VM setup time; taking up to 15 minutes from fresh start to completing installation of msys2 + clang.

Thanks for keeping this open.

[NilsIrl]

Would it be possible to have clang-tools as well (especially clang-format for our use case)?

[dg0yt]

Adding LLVM's clang-8 packages seems to have broken Ubuntu 18.04's clang-8 dependency chains. I have trouble with previously working tasks, such as apt-get install clang-tidy-8
~
The following packages have unmet dependencies:
clang-tidy-8 : Depends: libllvm8 (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed
Depends: clang-tools-8 but it is not going to be installed
~

or apt-get install libclang-8-dev:
~
The following packages have unmet dependencies:
libclang-8-dev : Depends: libclang1-8 (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed
Depends: libclang-common-8-dev (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed
~

[al-cheb]

Hello, @dg0yt
Could you please try?

sudo apt-get remove -y libllvm8
sudo apt-get install clang-tidy-8

[dg0yt]

@al-cheb The proposed code works for clang-tidy-8. I will test the other case later.
The removal of libllvm8 took away the following 10 packages:
~
clang-8 clangd-8 libclang-common-8-dev libclang1-8 liblldb-8 libllvm8 lld-8
llvm-8 llvm-8-dev llvm-8-runtime
~

[al-cheb]

@dg0yt,
As another workaround using llvm-toolchain-bionic-8 repo:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
REPO_NAME="deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main"
sudo add-apt-repository "${REPO_NAME}"
sudo apt-get update 
sudo apt-get install -t llvm-toolchain-bionic-8 clang-tidy-8

[GalAster]

Failed only on windos beacuse no LIBCLANG_PATH found: https://github.com/GalAster/carbon-rs/runs/492035660

Is there any way to save this?

'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'clang.dll\', \'libclang.dll\']
set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found

[splix]

@GalAster it seems you need to install LLVM for Windows, otherwise I had the same error for Rust.

Fixed with the following step:

 - name: Instal LLVM
   run: choco install -y llvm
   if: runner.os == 'Windows'

[adwhit]

I found scoop is faster than choco. I have the following:

      - name: Install LLVM (Windows)
        if: runner.os == 'Windows'
        shell: powershell
        run: |
          Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
          scoop install llvm --global

Note you'll maybe need to pass in

        env:
          LIBCLANG_PATH: "C:\\ProgramData\\scoop\\apps\\llvm\\current\\bin"

to subsequent actions.

[Lewiscowles1986]

does anyone here know how to force clang to build for mac?

[al-cheb]

@Lewiscowles1986, Could you please take a look - https://github.com/actions/virtual-environments/issues/658#issuecomment-611921753

[Lewiscowles1986]

Got it. I fixed last night by overriding CXX and CC environment variables. Thank you.

[aminya]

Clang is installed on windows-latest, but it is not on the PATH. Any solution? I even tried installing it using choco, but I got the same error.
https://github.com/aminya/include-what-you-use/runs/697682319?check_suite_focus=true#step:5:5

Is there an llvm-setup action? Installing multiple LLVM versions on different operating
systems is hard.

[Darleev]

@aminya Hello,
Currently, you can find clang/clang++ and other binaries, which possible absent in current MinGW installation, by paths:

x64
C:\msys64\mingw64\bin
x32
C:\msys64\mingw32\bin

In order to add them into the system PATH, please follow the steps below:

GitHub Actions:
echo "::add-path::C:\msys64\mingw64\bin"

Azure DevOps
echo '##vso[task.prependpath]C:\msys64\mingw64\bin'

In case of any questions, do not hesitate to contact us.

[Darleev]

In case of additional questions, do not hesitate to contact us, we will be glad to assist you further.

I'm going to close the issue.

[sbeyer]

In case of additional questions, do not hesitate to contact us, we will be glad to assist you further.

It's great that a lot of clang tools are available on windows-2019, however, clang-tidy is missing. Is there a reason for that? Is it going to be added?

[maxim-lobanov]

@sbeyer , in scope of this request, only base tools were installed. Feel free to create the separate request and we will consider adding clang-tidy

[advancedwebdeveloper]

I am curious if there any plans to support Clang 11.0 9 (and corresponding version of lld) ?