Virtual-environments: Update Xcode 11.4

[pixyzehn]

Tool information

• Tool name: Xcode
• Add or update? Update
• Desired version: 11.4 (11E146)
• Approximate size: 7.5 GB

Virtual environments affected

• [x] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [ ] Ubuntu 18.04 LTS
• [ ] Windows Server 2016 R2
• [ ] Windows Server 2019

Can this tool be installed during the build?

Are you willing to submit a PR?

No.

[miketimofeev]

Hi, @pixyzehn! Image with Xcode 11.4 will be rolled out during next week's image deployment. I will notify you when it's available.

[ydnar]

Apple is now rejecting uploads built with the current version of Xcode 11.4 on GitHub Actions (breaking our CD). Do you have an ETA for this deploy?

[miketimofeev]

@pixyzehn @ydnar the image will be updated this week. Sorry for the inconvenience.

[ydnar]

@miketimofeev thanks! We’re also looking forward to the deployment of Swift on Ubuntu (#609), which lets us run a number of Swift workflows on Linux instead of macOS without having to spend 40s installing Swift.

[andyhuber]

@miketimofeev
This situation is unfortunately quite severe for companies that are relying solely on GitHub Actions to upload builds. Our whole build pipeline is built on GitHub actions and that's the second time after 11.2 that we run into the same problems. Since we are doing nightly builds for all our inhouse testing we need this going.

Is there a possibility for future updates that you can have them ready when Apple is changing their acceptance criteria?

[ydnar]

@andyhuber strongly agree.

Possible to decouple Xcode from the normal Catalina / macOS build train?

[miketimofeev]

@andyhuber @ydnar sorry for the inconvenience. Image with Xcode 11.4 is rolling out right now and the will be delivered in a couple of hours. We are going to make such Xcode updates much faster in the future.
Thank you!

[ydnar]

Thank you! Appreciate your hard work right now.

[MaxDesiatov]

@miketimofeev does this include Azure Pipelines images with macOS 10.15? Could one assume that GitHub Actions images and Azure Pipelines images are now identical, given that the Azure Pipelines images repository redirects to this one?

[miketimofeev]

@MaxDesiatov Unfortunately, no. Azure pipelines pools are updated separately from GitHub ones. They will be updated tomorrow(you can follow this PR https://github.com/microsoft/azure-pipelines-image-generation/pull/1511, it will be merged when the deployment is done)

[stephencelis]

@miketimofeev Any idea how long it usually takes for these rollouts to complete? It looks like the current environment is still on the beta.

[andyhuber]

@stephencelis it seems to be updated by now.
@miketimofeev Thanks. The documentation https://github.com/actions/virtual-environments/blob/master/images/macos/macos-10.15-Readme.md
seems to still be outdated. Still mentioning 11.4 beta

[miketimofeev]

@stephencelis @andyhuber it's not yet updated for all the environments. The PR will be completed once it's done.
Thank you for your patience!

[miketimofeev]

@pixyzehn @andyhuber @stephencelis image rollout finished. Closing this issue, but feel free to reopen if you have any concerns.
@MaxDesiatov Azure pools will be updated a bit later today.
Thank you!

[arek-avanade]

@miketimofeev Something is not quite right with XCode 11.4 rollout. I'm seeing 2 different images in Azure Pipelines for macos-latest:

Agent name: 'Azure Pipelines 3'
Agent machine name: 'Mac-1624'
Current agent version: '2.165.2'
Current image version: '20200328.1'

This one works as expected and contains XCode 11.4.

However there is also another version that's being queued from time to time:

Agent name: 'Hosted Agent'
Agent machine name: 'Mac-1383'
Current agent version: '2.165.2'
Current image version: '20200404.1'

This one seems to be newer, however does NOT seem to have XCode 11.4 installed. I'm getting the following error when running xcode-select:
xcode-select: error: invalid developer directory '/Applications/Xcode_11.4.app/Contents/Developer'

[miketimofeev]

@arek-avanade it looks like some of your builds are run on Mojave agents(current image version 20200404.1), where Xcode 11.4 is not presented, and sometimes on Catalina(current image version 20200328.1). Could you please check your agent specifications?

[arek-avanade]

@miketimofeev This is an excerpt from my YAML:

- job:
    pool:
      vmImage: macOS-latest
      demands:
      - xcode
      - Xamarin.iOS
      - Xamarin.Android

When saying macOS-latest, I'd expect to actually get Catalina all the time, as described in this article:
https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops

[miketimofeev]

So it's the same pipeline, which sometimes triggered on 20200328.1 and sometimes on 20200404.1, right?
If so, could you please provide examples? If it's not possible then a plan-id for both builds will be helpful. It can be found by the following link:
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}

[arek-avanade]

Yes, the same pipeline, however different stages/jobs. So the failing scenario is for a job which is declared with pool/demands as above, it picks 20200404.1 image.
77a0d715-1fcf-47c1-9767-dd1b1329ed5b

The successful scenario is when I just specify vmImage: macOS-latest, but NO demands. In that case somehow 20200328.1 is picked. I'm not sure if this is a coincidence but it consistently happened a few times.
75bfcc08-76ad-4346-88b3-8c8d54cb5eea

[miketimofeev]

Thanks, but unfortunately, I don't have access to the builds. Would you mind providing plan-id for them?

[arek-avanade]

Oh, my bad. Didn't understand your previous ask. I've updated my previous answer with ids.

[miketimofeev]

Thanks! Yeah, one build is queued against 10.14 while another against 10.15
And what are those demands that you mentioned? I will try to repro it if I have the exact specifications.

[miketimofeev]

And I've found one important note

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/demands?view=azure-devops&tabs=yaml
Looks like it's better to avoid using demands at all.

[arek-avanade]

The one that triggers 10.14 and fails has:

pool:
      vmImage: macOS-latest
      demands:
      - xcode
      - Xamarin.iOS
      - Xamarin.Android

The one that triggers 10.15 has just:

pool:
      vmImage: macOS-latest

[arek-avanade]

And I've found one important note

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/demands?view=azure-devops&tabs=yaml
Looks like it's better to avoid using demands at all.

Oh, didn't know that. I'll simplify my YAML and try again in that case. Although I think this should still not drive the selection of agent image IMHO.

[miketimofeev]

I'm able to repro it with your demands. I'll ask the engineering team about that strange behavior, but yeah, please avoid using demands for now.

[maxim-lobanov]

@arek-avanade , yep, that is it.
All images have Xcode and Xamarin tools installed so we don't need to specify demands. Could you please try to remove them and test again?

[arek-avanade]

Removed 'demands' and now it works as expected. Thanks for the quick feedback and resolution guys!

[ydnar]

I'm able to repro it with your demands. I'll ask the engineering team about that strange behavior, but yeah, please avoid using demands for now.

A parable for our time.

[darthwillis]

There is a bit of legacy around custom user capabilities and demands for hosted pools that results in this behavior. We have some work on the backlog to make it better, but it hasn't been prioritized yet. Essentially, when demands are presented, we assign the job to the most relevant legacy hosted pool (i.e. one of the pools that existed before the single hosted "Azure Pipelines" pool).

If the YAML specifies a pool by name, then it will not redirect. So you could take your original YAML and add name: Azure Pipelines to the pool specification, and it would get to the right place like so:

- job:
    pool:
      name: Azure Pipelines
      vmImage: macOS-latest
      demands:
      - xcode
      - Xamarin.iOS
      - Xamarin.Android

This blog post covers a bit more details:
https://wouterdekort.com/2020/01/14/moving-to-the-azure-pipelines-agent-pool/