Runtime: Need GCC build pipeline

[franksinankaya]

We have been waiting for a long time to add gcc build pipeline. I'm looking for help from the build team.

[jkotas]

This is the blueprint for how to do it: https://github.com/dotnet/runtime/issues/2394#issuecomment-580417123 (just replace Tizen w/ gcc).

[jkotas]

gcc is part of the build docker containers as of https://github.com/dotnet/dotnet-buildtools-prereqs-docker/pull/233 . So this just needs the second step (add one CI leg with gcc).

[am11]

Top level build scripts are now also equipped with --gcc or --gcc<major>.<minor> flags support, so it is now just a matter of adding gcc job in pipeline's build matrix. See https://github.com/dotnet/runtime/issues/3972#issuecomment-595489386.

./build.sh --gcc --subsetCategory coreclr
./build.sh --gcc --subsetCategory libraries
./build.sh --gcc --subsetCategory installer

cc @xiangzhai for possible MIPS{64} interest.

[jkotas]

@safern Could you please suggest the right place to add a single GCC CoreCLR build leg to CI pipelines?

[franksinankaya]

@safern AFAIK, all tools needed to build with gcc and g++ are present in the docker container. We need to add a CI using the commands mentioned above from @am11 .

Can you help?

[safern]

I would imagine this as needing to add another block here:
https://github.com/dotnet/runtime/blob/master/eng/pipelines/runtime.yml#L125

Something like:

- template: /eng/pipelines/common/platform-matrix.yml
  parameters:
    jobTemplate: /eng/pipelines/coreclr/templates/build-job.yml
    buildConfig: checked
    platforms:
    - Linux_x64
    jobParameters:
      testGroup: innerloop
      useGCC: true
      condition: >-
        or(
          eq(dependencies.checkout.outputs['SetPathVars_coreclr.containsChange'], true),
          eq(variables['isFullMatrix'], true))

Note the useGCC: true that I added at the jobParameters section.

Then in coreclr/templates/build-job.yml, you would need to add a new parameter:
useGCC: false in the parameters section.

Then in the variables section for coreclr/templates/build-job.yml you would need to add a new variable for that argument, something like:

   - name: gccArg
      value: ''
    # Strip symbols only on the release build
    - ${{ if eq(parameters.useGCC, true) }}:
      - name: gccArg
        value: '-gcc'

Then, at the end you would need to use that arg when calling the build scripts as we do with the other args in the right place: https://github.com/dotnet/runtime/blob/fef9ca70f6e7268dfc9e0f600433f5293932f166/eng/pipelines/coreclr/templates/build-job.yml#L134

Also, I would recommend changing the display name for the CI build so that it reflects that it is using the GCC compiler.

That you can do by conditioning: https://github.com/dotnet/runtime/blob/fef9ca70f6e7268dfc9e0f600433f5293932f166/eng/pipelines/coreclr/templates/build-job.yml#L36

To something like:

  # Compute job name from template parameters
   ${{ if eq(parameters.useGCC, true) }}:
      name: ${{ format('coreclr_gcc_product_build_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
      displayName: ${{ format('CoreCLR GCC Product Build {0}{1} {2} {3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
   ${{ if eq(parameters.useGCC, false) }}:
    name: ${{ format('coreclr_product_build_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
    displayName: ${{ format('CoreCLR Product Build {0}{1} {2} {3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}

Do we only want to build coreclr with GCC? If we want to add a build of all the components using GCC, maybe it is better to just add a new leg of the runtime-live-build pipeline instead, which builds the whole runtime repo using the build.cmd script from the root.If that is the case please let me know to help with instructions on how to add a new leg for that pipeline instead.

[franksinankaya]

We want to build all.

[safern]

Ok, I was following @jkotas comment:

@safern Could you please suggest the right place to add a single GCC CoreCLR build leg to CI pipelines?

Then what you need to do is something similar but a little bit simpler:

You need to add a new block at the end of this file:
https://github.com/dotnet/runtime/blob/fef9ca70f6e7268dfc9e0f600433f5293932f166/eng/pipelines/global-build.yml#L85

Something like:

#
# Build using CoreCLR runtime and GCC as the compiler toolset.
#
- template: /eng/pipelines/common/platform-matrix.yml
  parameters:
    jobTemplate: /eng/pipelines/common/global-build-job.yml
    buildConfig: debug
    platforms:
    - Linux_x64
    jobParameters:
      testGroup: innerloop
      nameSuffix: Runtime_Debug_GCC
      buildArgs: -c Release -runtimeConfiguration Debug -gcc

Note the nameSuffix and buildArgs difference.

[jkotas]

We want to build all.

Building of stuff outside of coreclr via gcc is covered by Mono. I do not think it is required in CI.

[franksinankaya]

oh, ok. I didn't know that. We can focus on the previous instructions then.

[safern]

Note that I might have missed something that might show up when testing as yml is sometimes complicated hard to eye spot errors, specially with the complicated system we have. So please cc me in the PR and I can help with errors and any other questions.

[am11]

@franksinankaya, I can also help if you want to lead the way on this one. Please feel free to ping me here or gitter. :)

[franksinankaya]

@am11 go ahead and take it. I don't mind as long as it is done. Tracking gcc bugs are getting tiring.