Azure-rest-api-specs: App Service: Deployment Validation Failure on Nonexistent App Service Plan

[aholler2]

EDIT: This failure with ARM template validation has spread to resource groups created in every region.

Our team publishes templates for others to digest. Part of this process involves validating the templates before moving onto provisioning steps. Our pipelines worked fine up until a few weeks ago when app service template validation began to fail with the following API and with the following error:
https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/App-Service-20200102.2/providers/Microsoft.Resources/deployments/{deploymentName}/validate?api-version=2017-05-10

[error]The template deployment 'app-service-20200108-192835-2552' is not valid according to the validation procedure. The tracking id is 'a7227c1b-a472-4854-bf66-19a33eebe689'. See inner errors for details.

[error]Details:

[error]ValidationForResourceFailed: Validation failed for a resource. Check 'Error.Details[0]' for more information. [{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]

[error]Task failed while creating or updating the template deployment.

The validation of the app service should not require an app service plan to exist. For instance, app service slots do not require an app service to exist in order for the template to pass validation.

[aholler2]

Here's proof of app service validation being broken only in South Central US.

PowerShell Core 6 script that shows failure of app service validation in South Central US.
AppServiceValidationFailure.txt

[sbalakrishnanbe]

I'm getting the same error message for West Europe and North Europe region as well.

2020-01-15T11:57:34.8890998Z ##[error]ValidationForMultipleResourcesFailed: Validation failed for multiple resources. Check 'Error.Details[]' for more information. [{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."},{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]
2020-01-15T11:57:34.8891708Z ##[error]Task failed while creating or updating the template deployment.

[RudyCo]

Same for me!
https://stackoverflow.com/questions/59793978/create-a-new-devops-project-python-function-fails-with-serverfarmnotfound-erro

Very bad for Microsoft as I created my Azure Python Function from scratch from the Azure Portal (new DevOps project)

[aholler2]

Same for me!
https://stackoverflow.com/questions/59793978/create-a-new-devops-project-python-function-fails-with-serverfarmnotfound-erro

Very bad for Microsoft as I created my Azure Python Function from scratch from the Azure Portal (new DevOps project)

@RudyCo's developer community feedback item: https://developercommunity.visualstudio.com/content/problem/892009/create-a-new-devops-project-python-function-fails.html

[aholler2]

This app service validation failure due to a non-existent app service plan now occurs in every region.

[cbre-jpetryk]

Also seeing this error in South Central US.

[rickvi1]

Same here! trying to deploy an Azure Function to West Europe.
When I re-deploy an existing function I don't see any ARM issues.

[se7vanj]

• East US2
• ARM Template deployed from DevOps Pipeline Task - Azure resource group deployment

Error:
"details":[{"code":"ValidationForResourceFailed","message":"Validation failed for a resource. Check 'Error.Details[0]' for more information.","details":[{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]}]}}

[opolo]

West Europe. Impossible to create new any function apps as well due to this issue. Nice... Must have been down for over a month now I guess.

[jayeshpatel6]

Anyone still have this issue?

[munseym]

Still having this issue.

[hrafnkel]

Same here

[Jeej]

Please fix this, it is really annoying because we use ARM deployments for our functions

[nissha4]

FYI I was getting the same error when I tried to deploy a new function app added to an old ARM template file. It seems like ARM templates used to allow us to get app service plan id using the following code

variables": { "services_app_service_plan_id": "[concat(resourceId('Microsoft.Resources/resourceGroups',parameters('main_resource_group_name')),'/providers/Microsoft.Web/serverfarms/',parameters('services_app_service_plan'))]"

but this format with the concat is no longer valid. you have to do the following to get the app service plan id

"services_app_plan_id": "[resourceId(parameters('main_resource_group_name'),'Microsoft.Web/serverfarms',parameters('services_app_plan'))]"
hope this helps

[aholler2]

@nissha4 That is unrelated to this issue. This issue is regarding template validation and the requirement of having an existing app service plan in order to pass validation.

[naveedaz]

The intention behind the recently added template validation is to help customers catch issues earlier as opposed to halfway through deployment. This is an Azure wide effort. The best of intentions, however, sometimes do not yield the best of results. The validation causing woes for customers is when a deployment template creates a site where properties.serverfarmId specified for the site is somehow deemed invalid. We enforce that the App Service Plan should either already exist or is also added as a resource in the template. The exception to that rule is App Service Plans with free or consumption plan SKUs since those can be created implicitly. Having said that there were a couple of bugs with the validation for some time:

1. We were enforcing explicit App Service Plans (properties.serverFarmId) even for consumption plans
2. Template Language Expressions (TLEs) were not properly handled and would also block.

The validations were rolled out by region and as we discovered issues, we tried to fix them ASAP. The bugs above and other minor bugs in the validations have been fixed but it might still be the case that we are missed some scenario. Please let us know what those scenarios are so we can address them. Also, let us know if the fixes do not work for some reason either. We would appreciate your help in making the validations more robust so that it helps and not impedes your path to success.

Also, this GitHub issue slipped from under our radar. We are changing our processes so we keep these issues at the forefront.

We apologize for the inconvenience.

[rhyscooper]

I have the same issue, when I deploy my template which includes a serverFarm resource along with a functions app service, I get the below error:
Validation failed for a resource. Check 'Error.Details[0]' for more information. [object Object]

If I comment out the functions app service and deploy it works fine, then add the functions app service back it still fails though so i'm not sure if it's the same scenario. The error message isn't very useful though so it might not be related.

[naveedaz]

@rhyscooper Can you share the correlation Id and approximate time when you ran into the issue? I'll be happy to take a look.

[rhyscooper]

@rhyscooper Can you share the correlation Id and approximate time when you ran into the issue? I'll be happy to take a look.

Sure, I've repro'd the issue just now so the time it occurred was 2020-04-10T08:10:57.4729875Z and the tracking id is '281f5386-02dd-44e4-9dfe-55dda560e01e'

Thanks

[naveedaz]

@rhyscooper I looked at your deployment template. The template creates an App Service Plan (aka server farm) in the East US 2 region and then a function app in the UK South region. However, the template specifies the function app to be associated with the server farm in East US 2 (look at properties.serverfarmId of the Microsoft.Web/sites resource in the template). We do not support associating an app with a server farm in a different region. Seems like you want to create the function app on a dedicated SKU ASP. You should create the ASP (server farm) in the same region as the function app. I hope this helps unblock you.

[rhyscooper]

@naveedaz Great spot! looks like someone added the location as hardcoded in the dev environment so it didn't work when we moved to UAT.

I've removed the hardcoded location and the template now validates correctly, I guess the error message could do with being more helpful though!

Thanks for your help.

[naveedaz]

I am glad that resolved the issue for you. I agree. The error that the "The specified app service plan was not found" is misleading and also does not call out the App service plan that is missing. We will fix that asap. Thanks for the feedback.

Preflight Response: {"Status":"Failed","Error":{"Code":"ValidationForResourceFailed","Message":"Validation failed for a resource. Check 'Error.Details[0]' for more information.","Target":null,"Details":[{"Code":"ServerFarmNotFound","Message":"The specified app service plan was not found.","Target":null,"Details":null}]}}

[aholler2]

@naveedaz How can we validate that an app service template is valid without actually provisioning an app service plan at this point?

As I mentioned in the original issue, we had pipelines that build/release ARM templates for general consumption. These pipelines do a fail-fast template validation on all templates and then finally do template deployments. Now with this app service change we are going to have to do app service plan validation -> app service deployment -> all other template validation -> all other deployment. Instead of doing that, we're better off not doing app service validation at all.

No other resources have this hard requirement on other/parent resources in validation.

[haacked]

I'm running into this issue as well with westus2. I ran the command just now. Tracking Id: 7611e46c-c164-487b-85fa-e3602b90f155.

It returns The specified app service plan was not found.. But if I run the same command with a different appServicePlanLocation I get a message,

The resource 'serious-bots-plan' already exists in location 'westus2' in resource group 'serious-bots'. A resource with the same name cannot be created in location 'westus'. Please select a new resource name.

[vllama]

This issue is occurring as well in Canada Central.
The tracking id is 'ecbdbdbd-67a2-46ef-8c4b-26a38ca40197'

{'additional_properties': {}, 'code': 'InvalidTemplate', 'message': "Deployment template validation failed: 'The resource 'Microsoft.Web/serverfarms/ASP-NAME-test2' is not defined in the template. Please see https://aka.ms/arm-template for usage details.'.", 'target': None, 'details': None, 'additional_info': [<azure.mgmt.resource.resources.v2019_07_01.models._models.ErrorAdditionalInfo object at 0x04DDA750>]} 

if You have the app service defined but with
"condition": false
The error is as follows:

{'additional_properties': {}, 'code': 'InvalidTemplateDeployment', 'message': "The template deployment 'deployment_dry_run' is not valid according to the validation procedure. The tracking id is 'd53f5a74-393f-4a37-953b-09cfd9a4f7ad'. See inner errors for details.", 'target': None, 'details': [<azure.mgmt.resource.resources.v2019_07_01.models._models.ErrorResponse object at 0x05697970>], 'additional_info': None}   

[jebalert]

Having a similar issue when I attempt to deploy a new bot web app / channel registration to an existing app service plan.

{'additionalProperties': {}, 'code': 'InvalidTemplateDeployment', 'message': "The template deployment 'TestBot' is not valid according to the validation procedure. The tracking id is 'cd282e74-c315-4734-aee0-30126022bd5b'. See inner errors for details.", 'target': None, 'details': [{'additionalProperties': {}, 'code': 'ValidationForResourceFailed', 'message': "Validation failed for a resource. Check 'Error.Details[0]' for more information.", 'target': None, 'details': [{'additionalProperties': {}, 'code': 'ServerFarmNotFound', 'message': 'The specified app service plan was not found.', 'target': None, 'details': None, 'additionalInfo': None}], 'additionalInfo': None}], 'additionalInfo': None}

I know the service plan exists. Issue is occurring for East US. I've confirmed all applicable resources exist in East US.

[shuai-niki-yuan]

I run into similar issue as well when doing the template deployment to create resources including a function app: could someone help me check what might be wrong? the tracking id is: 466c1dbf-ebc0-4290-843b-0fb980848eec'
2020-06-04T23:02:58.0047618Z ##[error]The template deployment 'resources-20200604-230255-62a6' is not valid according to the validation procedure. The tracking id is '466c1dbf-ebc0-4290-843b-0fb980848eec'. See inner errors for details.
2020-06-04T23:02:58.0062175Z ##[error]Details:
2020-06-04T23:02:58.0064884Z ##[error]ValidationForMultipleResourcesFailed: Validation failed for multiple resources. Check 'Error.Details[]' for more information. [{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."},{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."},{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."},{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]

[naveedaz]

@aholler2 You do not need to actually create the resources to validate the template. Any template can be validated by calling this API. https://docs.microsoft.com/en-us/rest/api/resources/deployments/validate.

[naveedaz]

@haacked Thank you for reporting the issue. This issue is hit when the serverfarmId is not a complete ARM resource Id of the format "/subscriptions/{subId}/resourcegroups/{rg}/providers/microsoft.web/serverfarms/{sf}". We were not properly deducing the subscription and resource group from the site resource specified in the template. The fix is part of the next release. From the logs I see that you switched to the complete ARM resource Id format for the ASP and were able to unblock yourself.

[haacked]

Thanks @naveedaz, I tried the full ID and that didn’t work for me at the time. I ended up manually creating it to unblock myself. At least that’s how I remember it. 😝

[naveedaz]

@vllama The issue you are experiencing is not the same. From the logs, I do not see any call that gets to the Azure App Services RP.

[mayank-chaturvedi]

Hi @naveedaz . Facing a similar isse while trying to deploy a new bot web app / channel registration to an existing app service plan..
The error is as follows :

Azure Error: InvalidTemplateDeployment
Message: The template deployment 'SaxoT-Rex' is not valid according to the validation procedure. The tracking id is 'dbbba0ba-94f5-4e88-8042-aa588f59c2d5'. See inner errors for details.
Exception Details: Error Code: ValidationForResourceFailed . Message: Validation failed for a resource. Check 'Error.Details[0]' for more information.

Any help would be greatly appreciated.

[aholler2]

@aholler2 You do not need to actually create the resources to validate the template. Any template can be validated by calling this API. https://docs.microsoft.com/en-us/rest/api/resources/deployments/validate.

@naveedaz That's not true and is why I created this issue in the first place. The Test-AzResourceGroupDeployment cmdlet is using the deployments/validate REST API you mention. Validation of app service should not need an existing app service plan, however it does as you can see by my logs below. Validation of app service fails w/o an existing app service plan.

Test-AzResourceGroupDeployment -ResourceGroupName ahollerRG -TemplateFile "C:\Users\hollea\Documents\AzureTemplates\AzureArtifacts\templates\app-service\app-service.json" -TemplateParameterFile "C:\Users\hollea\Documents\AzureTemplates\AzureArtifacts\templates\app-service\app-service.param.json"


Code    : InvalidTemplateDeployment
Message : The template deployment '9bbc94d7-d8f6-4913-8efa-4675b46332a0' is not valid according to the validation procedure. The tracking id is 'cdcc43c7-d62f-4b27-b78c-68977d54a37c'. See 
          inner errors for details.
Details : {Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.PSResourceManagerError}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/subIdxxx/resourcegroups/ahollerRG/providers/Microsoft.Resources/deployments/d5d13c52-8799-4990-92f7-0af5b282bebc/validate?api-version=2019-10-01

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Pragma                        : no-cache
x-ms-failure-cause            : gateway
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-request-id               : 481d57fc-0038-4dcd-bef8-cbddce59cc3d
x-ms-correlation-request-id   : 481d57fc-0038-4dcd-bef8-cbddce59cc3d
x-ms-routing-request-id       : NORTHCENTRALUS:20200605T172326Z:481d57fc-0038-4dcd-bef8-cbddce59cc3d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Cache-Control                 : no-cache
Date                          : Fri, 05 Jun 2020 17:23:26 GMT

Body:
{
  "error": {
    "code": "InvalidTemplateDeployment",
    "message": "The template deployment 'd5d13c52-8799-4990-92f7-0af5b282bebc' is not valid according to the validation procedure. The tracking id is '481d57fc-0038-4dcd-bef8-cbddce59cc3d'. 
See inner errors for details.",
    "details": [
      {
        "code": "ValidationForResourceFailed",
        "message": "Validation failed for a resource. Check 'Error.Details[0]' for more information.",
        "details": [
          {
            "code": "ServerFarmNotFound",
            "message": "The specified app service plan was not found."
          }
        ]
      }
    ]
  }
}

[naveedaz]

@aholler2 We are re-evaluating serverFarmId validation on the site resource. Just so I understand the scenario completely, can you re-iterate why the server farm resource itself does not exist in template? Do you have linked templates?

[aholler2]

@aholler2 We are re-evaluating serverFarmId validation on the site resource. Just so I understand the scenario completely, can you re-iterate why the server farm resource itself does not exist in template? Do you have linked templates?

@naveedaz We do not utilize linked templates. Our team's current philosophy is to decouple and simplify templates whenever possible. Our team generalizes and publishes ARM templates for app team consumption in pipelines and in such a case the templates are not to be coupled and complex. This is intended to enable ease of use and different solution varieties. We have a simple template for creating an App Service plan. Then we have a separate non-linked template for deploying individual app service instances into that previously provisioned App Service plan. This lets us easily deploy one or more App Services into the plan/serverFarm based on the requirements of a specific deployment architecture.

However, when we're provisioning resources we want to be able to verify that all of the resources are valid (or as reasonably valid as possible) before we actually deploy or redeploy any resources. Essentially, we want to be able to deploy resources in a transactional manner and we don't want to get halfway through deploying resources and then find out that the deployment will fail. Take a case where we're deploying an App Service plan with three individual App Service instances. We don't want to deploy the App Service plan and the first two App Service instances before finding out that the third App Service will fail causing the application to be in a potentially unusable state. We want to be able to perform a 'pre-deployment' validation of those resources individually to have some level of confidence that the deployment will likely succeed.

[lukecolburn]

@naveedaz We use linked templates, and we discovered this issue when we ran a deployment that was provisioning a new app service plan instead of pointing to an existing one. If we run the linked template on its own and then run the parent template the validation passes and the template deploys fine. Is there a particular workaround for linked templates?

[Mjiduser]

Hello,

Is there any news /workaround for the use case regarding Linked templates? I am getting the same issue :

2020-06-21T11:54:41.2454441Z There were errors in your deployment. Error code: InvalidTemplateDeployment.
2020-06-21T11:54:41.2505894Z ##[error]The template deployment 'template-202006' is not valid according to the validation procedure. The tracking id is ''. See inner errors for details.
2020-06-21T11:54:41.2524729Z ##[error]Details:
2020-06-21T11:54:41.2527761Z ##[error]ValidationForResourceFailed: Validation failed for a resource. Check 'Error.Details[0]' for more information. [{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]
2020-06-21T11:54:41.2531775Z ##[warning]Validation errors were found in the Azure Resource Manager template. This can potentially cause template deployment to fail. Task failed while creating or updating the template deployment.. Please follow https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/template-syntax
2020-06-21T11:54:41.2533767Z Starting Deployment.
2020-06-21T11:54:41.2534312Z Deployment name is template-202006
2020-06-21T11:54:45.7463538Z There were errors in your deployment. Error code: InvalidTemplateDeployment.
2020-06-21T11:54:45.7466698Z ##[error]The template deployment 'template-2020062' is not valid according to the validation procedure. The tracking id is ''. See inner errors for details.
2020-06-21T11:54:45.7512600Z ##[error]Details:
2020-06-21T11:54:45.7517660Z ##[error]ValidationForResourceFailed: Validation failed for a resource. Check 'Error.Details[0]' for more information. [{"code":"ServerFarmNotFound","message":"The specified app service plan was not found."}]
2020-06-21T11:54:45.7520440Z ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
2020-06-21T11:54:45.7522593Z ##[error]Task failed while creating or updating the template deployment.

[tyomik-mnemonic]

Hello! Face a similliar problem. There is my deployment trying:
az deployment create --template-file "\tmkpy\botbuilder-samples\generators\python\app\templates\echo\{{cookiecutter.bot_name}}\deploymentTemplates\template-with-new-rg.json" --location germanywestcentral --parameters appId="xxxx" appSecret="xxxx" botId="Xxx" botSku=F0 newAppServicePlanName="PyBotServ" newWebAppName="PyBotWeb" groupName="PyBotGroup" groupLocation="Germany West Central" newAppServicePlanLocation="Germany West Central" --name "PyBot"
and console throws it
'additionalProperties': {}, 'code': 'InvalidTemplate', 'message': "Deployment template validation failed: 'The resource 'Microsoft.Web/serverfarms/PyBotServ'

Same with Central US

[gregvinyard]

Getting the same error,
InvalidTemplate - Deployment template validation failed: 'The resource 'Microsoft.Web/serverfarms/myBotServicePlan' is not defined in the template. Please see https://aka.ms/arm-template for usage details.'.

I'm using the template-with-new-rg.json file that was included in the sample bot app I downloaded from azure.

Edit: So, it looks like the issue is there is no "Microsoft.Web/serverfarms" resource defined in the template-with-new-rg.json file.
Edit 2: Also no "Microsoft.Web/sites" resource.
Edit 3: And I get the same error using https://github.com/microsoft/BotBuilder-Samples/blob/master/samples/csharp_dotnetcore/02.echo-bot/DeploymentTemplates/template-with-new-rg.json

[bowlerma]

I don't suppose there's an update on this?

As others have pointed out; only occurs when the service plan isn't defined in the same template as the app service. Nothing else seems to require that a resource actually exists when using the 'validate' mode.

[aholler2]

I don't suppose there's an update on this?

As others have pointed out; only occurs when the service plan isn't defined in the same template as the app service. Nothing else seems to require that a resource actually exists when using the 'validate' mode.

@naveedaz?

[aholler2]

It seems this issue may have been resolved or rolled back by Azure. I cannot reproduce this "ServerFarmNotFound" error anymore. Can you confirm if it got fixed @naveedaz?