Meson: Move some of our appveyor jobs over to Azure

[nirbheek]

https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/

This should speed up our CI massively. We can probably move over our VS2015 and VS2017 jobs:

VS2017 image spec: https://github.com/Microsoft/vsts-image-generation/blob/master/images/win/Vs2017-Server2016-Readme.md

VS2015 image spec: https://github.com/adventworks/hosted-pool-images/blob/2017.10.02/vs2015-on-windows-2012r2/image.md

Possibly others too:

All images: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=vsts&tabs=yaml

[nirbheek]

Appveyor: https://www.appveyor.com/docs/build-environment/#pre-installed-software

Mono, LLVM, MSYS2, Cygwin, Qt, Boost are missing in Azure but we only need one appveyor job that runs with VS 2017 for those.

[lazka]

I've played around with it a bit and created an example MSYS2 repo: https://github.com/lazka/msys2-azure-pipelines

Build logs: https://lazka.visualstudio.com/test2/_build/results?buildId=36&view=logs

(probably not that useful, but so you can see how things look like)

[lazka]

Two lessons learned so far:

• Never install Azure through the GitHub Marketplace and go through https://dev.azure.com instead and set it up through OAuth. Otherwise all links to build logs on GitHub require a Microsoft login.
• The branch trigger for branches (not master, PRs) are broken by default and setting in yaml doesn't work. The workaround I found so far is going to the pipeline settings in the web UI > triggers > change "Branch specification" from master to *

[nirbheek]

Never install Azure through the GitHub Marketplace and go through https://dev.azure.com instead and set it up through OAuth. Otherwise all links to build logs on GitHub require a Microsoft login.

Oh my god, @jpakkane I'm sorry I made you install that, do you think you could uninstall it?

[jpakkane]

Removed.

[lazka]

And one more: any kind of GUI doesn't work (gtk will fail for example), see https://docs.microsoft.com/en-us/azure/devops/pipelines/test/set-up-continuous-test-environments-builds?view=vsts#q-can-i-run-ui-tests-on-the-microsoft-hosted-agents Might not be relevant for meson.

[chrispat]

I @lazka I am a PM on the Azure Pipelines team

That document is out of date. The VS2017 images are running in an interactive desktop environment so you should be able to run GUI tests there. The Ubuntu environment as a virtual frame buffer installed so you should be able to export a display on those.

The app is the preferred way to integrate as you get the Checks instead of just PR status. The issue you ran into was a bug that we are working to resolve.

Please feel free to ping me if you have any questions.

[nirbheek]

Thanks @chrisrpatterson, we'll probably have questions/suggestions once we start moving our pipelines over. One thing we'd like to see is more software pre-installed on the images (Qt, Boost, LLVM, Cygwin, etc) but it's not a blocker for us to move. It'll just speed up the jobs.

[lazka]

That document is out of date. The VS2017 images are running in an interactive desktop environment so you should be able to run GUI tests there. The Ubuntu environment as a virtual frame buffer installed so you should be able to export a display on those.

@chrisrpatterson Oh, that's great to hear, thanks! I was testing with the vs2015 one :)

edit: works nicely with vs2017

[jeremyepling]

@nirbheek / @lazka , we're building a list of tools to add to the pre-installed images. I created an issue for the ones you need.

[jon-turney]

I made an attempt at this in #4340. Some issues, in no particular order:

• Checkout is on D:, temporary files on C: in this environment, which tickles bug #3239
• The behaviour test_find_program() exercises seems to vary depending on if PATHEXT contains .py, which is not the case in this environment (#4355).
• With the vs2015 image, installers fail with an error about requiring an interactive window station
• python3 is apparently installed in the vs2015 image, but not on PATH and I don't know where...
• The git checkout is made with autocrlf true, and I don't see a way to change that. So line-ending checks on source code need to be disabled.
• Unclear what forms wildcards in trigger: can take, and how to match our release branch names
• Status badge needs wiring up
• [ci skip] is spelt differently?

[jpakkane]

Most of those seem to be limitations of the system (such as autocrlf, paths and installers) that should be fixed upstream. @jeremyepling?

[jeremyepling]

Let me get someone else on the team to help with the first 3 issues from @jon-turney. I answered the other below.

• This doc has info on how to use a specific version of Python
• You can create a .gitattributes file to force LF line endings.
• I'll update the triggers doc so it's more clear. They support prefix matches only, such as release* for release-1 and release2. What is your release branch format? Do you have a common prefix?
• [ci-skip] should be an easy fix. I'll see if we can sneak it into this sprint's deployment.

[chrispat]

• Checkout on D is to provide more consistent IO performance for the job. The D: drive is on the local SSD for the VM rather than remote storage
• I am not sure about the test_find_program() issue or exactly what is causing that.
• The VS2017 image runs in an interactive session and has multiple versions of python. Can you use that image?

[jon-turney]

My first two bullet points are meson bugs. Sorry for being unclear about that. I've made workarounds for them in .azure-piplelines.yml.

• The VS2017 image runs in an interactive session and has multiple versions of python. Can you use that image?

meson claims to support vs2010, vs2015 and vs2017. So ideally, we would be able to test with all those.

[nirbheek]

@jeremyepling

• [ci-skip] should be an easy fix. I'll see if we can sneak it into this sprint's deployment.

For consistency with Appveyor and Travis, it would be great if this would be [skip ci] so we can use the same tag for all CI :)

@jon-turney

meson claims to support vs2010, vs2015 and vs2017. So ideally, we would be able to test with all those.

I think it's a win even if we can only move some of our pipelines over, so if we can get vs2017 in first and add the rest later, that's good too!

[jon-turney]

Thank you both for your replies.

• This doc has info on how to use a specific version of Python

This didn't work for me with the vs2015 image. logs

##[error]Version spec >=3.5 for architecture x64 did not match any version in the tool cache.
Available versions:

The installed software list for the Visual Studio 2015 on Windows Server 2012r2 image says it contains 'Python 3.6.1', but I don't know how I'm supposed to find out the path for that.

• I'll update the triggers doc so it's more clear. They support prefix matches only, such as release* for release-1 and release2. What is your release branch format? Do you have a common prefix?

At the moment, the branches are just named after the major release number e,g. `0.48', with no prefix.

[jon-turney]

I think it's a win even if we can only move some of our pipelines over, so if we can get vs2017 in first and add the rest later, that's good too!

Absolutely. Just moving those 2 jobs will save ~20 minutes per run.

[brcrista]

Hey @nirbheek , Use Python Version is for working with multiple versions of Python side-by-side. The VS2015 doesn't have that capability. VS2017, Hosted macOS, and Hosted Ubuntu 16.04 all have it.

The Python 3.6.1 from that doc is the one installed by Visual Studio. It should be in PATH by default and you don't need to use the task.

[jon-turney]

The Python 3.6.1 from that doc is the one installed by Visual Studio. It should be in PATH by default and you don't need to use the task.

@brcrista I'm obviously missing something here. Adding the following to my build steps:

where python
where python3

gives me this with the vs2015 image

2018-10-10T17:13:15.5312299Z C:\Python27\python.exe
2018-10-10T17:13:15.7690134Z INFO: Could not find files for the given pattern(s).

logs

[brcrista]

@lokesh755 @vtbassmatt can you help?

[vtbassmatt]

Hey all. Do you need Python3 on the VS 2015 image? That image is basically sealed -- we aren't adding new stuff to it. I'd suggest running your Python 2.7 steps there (I know there's a VC9 dependency) and hosting everything else on VS2017.

[nirbheek]

Meson requires Python 3.5 or newer, so we can't use the VS2015 image unless that's installed. I suppose we can always install it as part of the build steps with msiexec?

[vtbassmatt]

You can install with msiexec. We have Python 3.6 on the image though as you're finding, something's not quite right. We're investigating that one, just thought I'd mention that you could switch to the VS2017 image for much better and more precise control of your Python version.

[jon-turney]

I suppose we can always install it as part of the build steps with msiexec?

python.org doesn't provide a .msi installer for python >3.4