Azure-pipelines-tasks: AzureFileCopy VSTS Task - Invalid Api Version

[anevjes]

As of today - the out of the box FileCopy VSTS task is throwing invalid API version ERROR. Most likely the Azure Storage API was changed in the actual task....

018-04-24T05:06:11.6526379Z Task : Azure File Copy
2018-04-24T05:06:11.6526557Z Description : Copy files to Azure blob or VM(s)
2018-04-24T05:06:11.6526740Z Version : 1.0.112
2018-04-24T05:06:11.6526902Z Author : Microsoft Corporation
2018-04-24T05:06:11.6527127Z Help : More Information
2018-04-24T05:06:11.6527337Z ==============================================================================
2018-04-24T05:06:16.2842604Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM\2.1.0\AzureRM.psd1 -Global
2018-04-24T05:06:33.9299758Z ##[warning]The names of some imported commands from the module 'AzureRM.Websites' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2018-04-24T05:06:33.9359471Z ##[warning]The names of some imported commands from the module 'AzureRM' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2018-04-24T05:06:33.9679257Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM.Profile\2.1.0\AzureRM.Profile.psm1 -Global
2018-04-24T05:06:34.0346657Z ##[command]Add-AzureRMAccount -ServicePrincipal -Tenant * -Credential System.Management.Automation.PSCredential -EnvironmentName AzureCloud
2018-04-24T05:06:36.5591098Z ##[command] Select-AzureRMSubscription -SubscriptionId XXXXXXXXXXXXXXXXXXXXXXXX -TenantId *
2018-04-24T05:06:40.9539800Z ##[error]InvalidApiVersionParameter : The api-version '2018-05-01' is invalid. The supported versions are '2018-02-01,2018-01-01,2017-12-01,2017-08-01,2017-06-01,2017-05-10,2017-05-01,2017-03-01,2016-09-01,2016-07-01,2016-06-01,2016-02-01,2015-11-01,2015-01-01,2014-04-01-preview,2014-04-01,2014-01-01,2013-03-01,2014-02-26,2014-04'.

[Vossekop]

Sooo many build and release definitions are going to fail today. We are having the same problem. I would have imagined some sort of testing is done before these things are updated in the marketplace.

[KrishnaAdityaB]

We faced the same issue in our automated prod test runs too.
Seems like Azure started throwing errors for some versions of some cmdlets.

We are working on verifying which versions are working. We will publish the list here.
And we will see if there is an alternative command that the AzureFileCopy task can use - and hotfix it.

[Vossekop]

Ok I am trying to find a work around at the moment, and noticed something peculiar. It says the latest installed version of the task is 1.0.113 but my release definitions are running 1.0.112.

[berendhaan]

Same issue here. Any ETA on a fix or a workaround?
Task verison: 1.0.112

[KrishnaAdityaB]

Right now, we are prioritizing to see which versions of Azure PS SDK are working.
Below is the list as of now

Not working:
1.0.7
1.3.2
2.1.0

Working:
4.4.1
5.7.0

[KrishnaAdityaB]

@Vossekop @berendhaan
The task update reaches accounts scale unit by scale unit. Your account might be in a scale unit which is last in the list.

I am sure the issue is not with Azure File Coyp task itself. So, I am deprioritizing this issu enow.
I will sync up with you later on this issue.

[Vossekop]

I would love to try it with Azure PS SDK version 4.4.1. But how do I instruct the task to use that version on my Hosted 2017 agent? It seems to use v2.1.0 by default.

[Vossekop]

Nevermind, I've added a powershell script to my release definition before the file copy task:

Install-Module -Name AzureRM -AllowClobber -Force

It installs 5.7.0, and this version also seems to work.

[tonesandtones]

+1

We're seeing exactly the same issue as originally described by @anevjes. An AZ File Copy task on a hosted vsts agent running az file copy 1.0.113 is failing, where previous builds using 1.0.112 succeeded normally.

[KrishnaAdityaB]

@Vossekop @berendhaan @tonesandtones
Regarding the task version.
The version in M132 release is 1.0.112
The version in M133 release is 1.0.113

We have deployed M132 to all scale units. M133 deployment is in progress. Your accounts will be updated to M133 and you should be getting newer version in a week or so.

Sometimes, you may notice, Build works with latest version while Release is having older version.
This happens because Task Definitions are stored with Build service. It may take a while before they are updated at Release server. But this should not be a problem in this case.

[KrishnaAdityaB]

@tonesandtones can you try with AzureRM PS SDK version 4.4.1 or the solution suggested by Vossekop?

[allanmoller]

Same here, worked fine yesterday and now this :-(

[tonesandtones]

Not sure I understand @krishnaadityab. It's the M133 release, using az file copy 1.0.113, that has the problem. Previous executions of the task used 1.0.112 and worked fine. This appears to be a regression in the M133 release.

[KrishnaAdityaB]

@tonesandtones
Nothing much was changed in AzureFileCopy task from 1.0.112 to 1.0.113.
Even 1.0.112 version failed for some users (above comments).

The Get-AzureRmResource cmdlet is failing with certain versions of Azure PowerShell even when tried manually in a PowerShell window.

I raised an issue with Azure PowerShell team https://github.com/Azure/azure-powershell/issues/6031

[tonesandtones]

@KrishnaAdityaB Understood, thanks for the explanation. If not resolved sooner, I'll try @Vossekop 's fix to install -clobber the AzureRM module soon. (I'm lucky enough that this is not a breaking issue for me)

[Konnor5092]

@Vossekop Are you able to share more specifics regarding the workaround?

I've got an Azure PowerShell task on version 2, set to 'Inline Script' where I have pasted 'Install-Module -Name AzureRM -AllowClobber -Force' into the inline script box. 'Azure Powershell Version' is set to 'Latest installed version'.

When running this with an on-prem agent I get 'Exception calling "ShouldContinue" with "2" argument(s): "Windows PowerShell is in NonInteractive mode. Read and Prompt functionality is not available."'

[Vossekop]

@Konnor5092 I've used this approach on the Hosted 2017 agent. I think these agents run in interactive mode so this might be why it is working for me. If you are using a private agent however, it might be easier to just RDP into the machine and install it manually (unless you have an extensive collection of agents or other complicating factors ofcourse).

[KrishnaAdityaB]

@Konnor5092 You may have to do Install-PackageProvider nuget -force before Install-Module like it was mentioned here

[Konnor5092]

@Vossekop Thanks. Running it on the build VM provided the workaround.

[nboettcher]

I'm having the same exact issue starting this morning.
Azure File Copy task v 1.0.113 and it is trying to to use PS 2.1

2018-04-24T12:49:51.0060843Z ##[section]Starting: Copy SQL Scripts to Azure Blob
2018-04-24T12:49:51.0065857Z ==============================================================================
2018-04-24T12:49:51.0066305Z Task : Azure File Copy
2018-04-24T12:49:51.0066525Z Description : Copy files to Azure blob or VM(s)
2018-04-24T12:49:51.0066748Z Version : 1.0.113
2018-04-24T12:49:51.0066954Z Author : Microsoft Corporation
2018-04-24T12:49:51.0067260Z Help : More Information
2018-04-24T12:49:51.0067533Z ==============================================================================
2018-04-24T12:49:56.6766877Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM\2.1.0\AzureRM.psd1 -Global
2018-04-24T12:50:16.1013718Z ##[warning]The names of some imported commands from the module 'AzureRM.Websites' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2018-04-24T12:50:16.1082138Z ##[warning]The names of some imported commands from the module 'AzureRM' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2018-04-24T12:50:16.1411099Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM.Profile\2.1.0\AzureRM.Profile.psm1 -Global
2018-04-24T12:50:16.2378749Z ##[command]Add-AzureRMAccount -ServicePrincipal -Tenant * -Credential System.Management.Automation.PSCredential -EnvironmentName AzureCloud
2018-04-24T12:50:18.3090841Z ##[command] Select-AzureRMSubscription -SubscriptionId 5d8a415c-3a5f-42c6-acc3-285d2b86d2ac -TenantId *
2018-04-24T12:50:22.5358009Z ##[error]InvalidApiVersionParameter : The api-version '2018-05-01' is invalid. The supported versions are '2018-02-01,2018-01-01,2017-12-01,2017-08-01,2017-06-01,2017-05-10,2017-05-01,2017-03-01,2016-09-01,2016-07-01,2016-06-01,2016-02-01,2015-11-01,2015-01-01,2014-04-01-preview,2014-04-01,2014-01-01,2013-03-01,2014-02-26,2014-04'.
2018-04-24T12:50:22.5971622Z ##[section]Finishing: Copy SQL Scripts to Azure Blob

[pbelousov]

It's good to have the workaround, but if we have tens definitions with this extension we will need to change all of them. And also this workaround significantly increases the execution time.

[KrishnaAdityaB]

The issue on Azure side was mitigated now. You can roll back your work-around.

@anevjes @Vossekop @berendhaan @tonesandtones @allanmoller @nboettcher @pbelousov