Azure-pipelines-tasks: VSTS gives FailedToUpdateApplicationInsightsResource code 201 on deploy Azure App Service

[jlcj1974]

Environment

• Server - VSTS

Issue Description

We are seeing a warning from app deploys originating with an Application Insights release annotation update. From the code in azure-arm-rest/azure-arm-appinsights.ts, the update method expects a 200 status code response and the documentation at https://docs.microsoft.com/en-us/rest/api/application-insights/components/createorupdate indicates that would be what is returned. However, we are getting a 201 status code which results in an error that AzureClient.ts emits with getFormattedError as "(CODE ${error.statusCode})":

public getFormattedError(error: any): string {
        if(error && error.message) {
            if(error.statusCode) {
                var errorMessage = typeof error.message.valueOf() == 'string' ? error.message 
                    : (error.message.Code || error.message.code) + " - " + (error.message.Message || error.message.message)
                error.message = `${errorMessage} (CODE: ${error.statusCode})`
            }

            return error.message;
        }

        return error;
    }

Error logs

2018-08-13T21:04:47.6668592Z Successfully deployed web package to App Service.
2018-08-13T21:04:50.2459456Z ##[warning]Can\'t find loc string for key: FailedToUpdateApplicationInsightsResource
2018-08-13T21:04:50.2464136Z Failed to add release annotation. Error: FailedToUpdateApplicationInsightsResource relode-apps undefined - undefined (CODE: 201)
2018-08-13T21:04:53.3416677Z Successfully updated deployment History at https://relode-api-staging.scm.azurewebsites.net/api/deployments/17861534194290237
2018-08-13T21:04:53.3510848Z ##[section]Finishing: Deploy API

[rickvdbosch]

We're having the same issue after updating our NuGet packages. Currently investigating if it has something to do with the restraint on Microsoft.ApplicationInsights.DependencyCollector version 2.7.1 which requires Microsoft.ApplicationInsights to be version 2.7.1. Currently we have version 2.7.2 installed.

Edit
The fact we're getting a 201 response instead of the expected 200 (which @jlcj1974 also pointed out) suggests the issue is outside of the packages, but in the API. Right?

Error logs
2018-08-15T01:05:56.8351185Z ##[warning]Can\'t find loc string for key: FailedToUpdateApplicationInsightsResource 2018-08-15T01:05:56.8361270Z Failed to add release annotation. Error: FailedToUpdateApplicationInsightsResource XXXXXXXX undefined - undefined (CODE: 201) 2018-08-15T01:06:03.7285149Z Successfully updated deployment History at https://xxxxxx.scm.azurewebsites.net/api/deployments/SOME_NUMBER

[pb-qs]

I am getting a similar problem... shown as a 400 response from the Annotations endpoint, with debug on...

2018-08-15T10:20:39.8002502Z ##[debug][PUT]https://management.azure.com/subscriptions/xxxx/resourceGroups/xxxx/providers/microsoft.insights/components/myappinsresource123/Annotations?api-version=2015-05-01'

2018-08-15T10:20:40.4916839Z ##[debug]addReleaseAnnotation. Data : {"statusCode":400,"statusMessage":"Bad Request","headers":{"cache-control":"no-cache","pragma":"no-cache","content-length":"36","content-type":"application/json; charset=utf-8","expires":"-1","strict-transport-security":"max-age=31536000; includeSubDomains","x-content-type-options":"nosniff","x-ms-ratelimit-remaining-subscription-writes":"1199","server":"Microsoft-IIS/8.5","x-ms-request-id":"xxxxx","x-ms-correlation-request-id":"xxxxx","x-ms-routing-request-id":"NORTHEUROPE:20180815T00xxxxx","date":"Wed, 15 Aug 2018 10:20:40 GMT","connection":"close"},"body":{"Message":"An error has occurred."}}

2018-08-15T10:20:40.4982899Z ##[warning]Can\'t find loc string for key: FailedToUpdateApplicationInsightsResource

2018-08-15T10:20:40.4993165Z ##[debug]Processed: ##vso[task.issue type=warning;]Can\'t find loc string for key: FailedToUpdateApplicationInsightsResource

2018-08-15T10:20:40.4993615Z Failed to add release annotation. Error: FailedToUpdateApplicationInsightsResource myappinsresource123 undefined - An error has occurred. (CODE: 400)

[SergeyKanzhelev]

CC: @frankguodongchen and @barustum in case it's error on Application Insights side, not in VSTS

[jlcj1974]

To clarify on our issue, we are receiving what looks like a HTTP 201 if I trace the error through the formatter correctly. 201 is a CREATED which is a success but the code above is looking for HTTP 200. https://httpstatuses.com/201

[SergeyKanzhelev]

@jlcj1974 did it work for you before? I wonder if something was changed recently

[jlcj1974]

The warning started for us with releases on Monday 8/13/2018.

[asranja]

It seems like the release annotations endpoint now returns a 201 instead of 200. @frankguodongchen and @barustum, please let us know if there has been any update recently so that we can fix the task accordingly.

[barustum]

I'll take a look.

Adding @jpiyali as FYI.

[jpiyali]

@asranja - Annotations always returned 201 and not 200 for creation of annotation. However, the response code is not returned from the VSTS task. So, not sure where you are using this for validation purposes.

[jpiyali]

I checked the code and there have not been any changes in this route for sometime. We saw a few failures with 400 but not for any create requests. Please provide x-ms-correlation-request-id to help with investigation.

[jlcj1974]

Where would I find that?

[htoma]

I'm seeing the same warnings when using VSTS to deploy Function Apps:

2018-08-17T09:46:46.3005386Z ##[warning]Can\'t find loc string for key: FailedToUpdateApplicationInsightsResource
2018-08-17T09:46:46.3021074Z Failed to add release annotation. Error: FailedToUpdateApplicationInsightsResource undefined - undefined (CODE: 201)

[asranja]

@jlcj1974 x-ms-correlation-request-id should not be needed for return code of 201.

[asranja]

@jlcj1974 It is quite odd that the annotations always returned a 201 on success. Our task had been working fine till now expecting a 200 return code from annotations.

[ffloimair]

Same problem here with VSTS WebApp deployment.
No changes were made on our side and this configuration was working without any flaws for almost a year.

[jlcj1974]

Sorry I am a bit confused as to what more you need. If the application insights endpoint is returning a 201 on success, it just seems like the VSTS task code in github needs to change from checking for 200 to 201.

Joe Christopher
Sent from my phone

On Aug 17, 2018 at 05:14,

@jlcj1974 (https://github.com/jlcj1974) x-ms-correlation-request-id should not be needed for return code of 201.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (https://github.com/Microsoft/vsts-tasks/issues/8022#issuecomment-413821735), or mute the thread (https://github.com/notifications/unsubscribe-auth/AC3TOR-us0F83cMZX8nxm2xOvCj7x6iTks5uRpd5gaJpZM4V7jcn).

[barustum]

We have identified a possible regression. A peer team is working on a fix.

[barustum]

To summarize:

• @jlcj1974 Yes the return code changed from 200 to 201, which is the correct response code. @asranja is making the fix in his script to account for the 201 response code.

• We are also fixing an issue in our backend code that may result in 400s. Fix will be in PROD next week

• @pb-qs I am not sure why you're getting the 400. Can you provide me with the resource name so I can look at our logs and check the failure for you.

[jlcj1974]

Thank you for your help!

On Aug 17, 2018 at 18:36,

To summarize:

@jlcj1974 (https://github.com/jlcj1974) Yes the return code changed from 200 to 201, which is the correct response code. @asranja (https://github.com/asranja) is making the fix in his script to account for the 201 response code.

We are also fixing an issue in our backend code that may result in 400s. Fix will be in PROD next week

@pb-qs (https://github.com/pb-qs) I am not sure why you're getting the 400. Can you provide me with the resource name so I can look at our logs and check the failure for you.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (https://github.com/Microsoft/vsts-tasks/issues/8022#issuecomment-414012892), or mute the thread (https://github.com/notifications/unsubscribe-auth/AC3TOVLGaO7_GvWU4VmYm4Xji3j5XkGOks5uR1N-gaJpZM4V7jcn).

[pb-qs]

The error message is being poorly described I think.

The problem might be related to the fact the Web app being registered with App Insights was using an App Svc Plan which was in a resource group that the agent didn't have permissions on. All withing locked VNETs with NSGs too.

When we added the VSTS agent account to the ASP's resource group as contributor the warning went away.

Many thanks,
Peter

Qualitas Software | e: [email protected]

-------- Original Message --------
Subject: Re: [Microsoft/vsts-tasks] VSTS gives FailedToUpdateApplicationInsightsResource code 201 on deploy Azure App Service (#8022)
From: "Basel Rustum" [e-mail]
Date: Sat, August 18, 2018 12:36 am
To: "Microsoft/vsts-tasks" [e-mail]
CC: "pb-qs" [e-mail],"Mention" [e-mail]

To summarize:

• @jlcj1974 Yes the return code changed from 200 to 201, which is the correct response code. @asranja is making the fix in his script to account for the 201 response code.

• We are also fixing an issue in our backend code that may result in 400s. Fix will be in PROD next week

• @pb-qs I am not sure why you're getting the 400. Can you provide me with the resource name so I can look at our logs and check the failure for you.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Microsoft/vsts-tasks/issues/8022#issuecomment-414012892

[SweetMeatPumpkin]

I got the same issue with 201 response code, but it doesn't seem to deploy.
Do you know when @asranja will fix this script to account for the 201 response code and push it to PROD?

[Davy-F]

Getting this too, but nothing was deployed apart from the function.json.

Have other peoples apps successfully been deployed with this error?

[asranja]

We have made the fix in the App Service task. It should take a few days for the fix to be rolled out on all the accounts.

[stylesm]

I am getting this with an Azure API App deployment. It is actually causing the entire package not to be deployed to the App Service, which returns a 500 because of this error.

[vincent1173]

@stylesm , Can you please share the debug logs (set variable system.debug to true and rerun the release)?

[stylesm]

@vincentdass I actually re-ran the deployment to the same environment / App Service and it worked fine. I haven't experienced this issue since, but I guess it is intermittent?

[vincent1173]

@stylesm , feel free to reopen / create a new thread if the issue occurs again.

[vincent1173]

Summary
Azure Application Insights REST API for create release annotation started returning 201 instead of 200 as status code. This caused the failure in VSTS task which expected 200. The fix is made to check for both the status code (200 & 201) as successful state.

[jpiyali]

As an fyi, appinsights reverted that change last week thursday to return 200 again. This will not break anything since the vsts task here checks for both 200 and 201.