Aspnetcore: Deployment fails due to file in use

[pekkah]

Any workaround for this problem when deploying to Azure web site from VSO build server vNext?

Error Code: ERROR_FILE_IN_USE
More Information: Web Deploy cannot modify the file 'AspNet.Loader.dll' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock
, or use the AppOffline rule handler for .Net applications on your next publish attempt.
Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.

I tried adding a restart azure website command before the deployment but this seems to be still creeping in too often.

[guardrex]

I noticed a similar issue with IIS file locking on Azure VM's that prevented WebDeploy deployment. My workaround is to issue three WebDeploy commands: stop the AppPool, deploy, restart the AppPool. See my script; and if you have the ability to run three WebDeploy commands, you might be able to do something similar with Azure Apps.

[Tazer]

Getting this problem in beta6.

[jochenvanwylick]

Same here! It causes my Visual Studio 2015 to freeze actually - when publishing my vNext app ( both beta6 and beta7 I think ).

[brandonmartinez]

Are you using the scripts provided by Microsoft to do the deployment? If so, you can just add a line in the PublishAspNet5Website.ps1 to restart the web app:

Restart-AzureWebsite -Name $websiteName

I have my build script modifications outlined in this post: Resolve an ASP.NET 5 Deployment Issue to Azure Web App Slot

[rubenprins]

@brandonmartinez, Restarting a website won't help, as a production website will keep receiving requests, which will therefore relock the files. The only reliable way, currently, to update a both IIS and Azure websites is to shut down the web site's application pool, then update the files, and then restart the website. Just follow @GuardRex's link for more details.

However, we shouldn't need to muck about with custom deployment scripts when working under default conditions with VS2015.

Hopefully the new HttpPlatformHandler infrastructure (which no longer needs an AspNet.Loader.dll) will be a bit more forgiving, but I'm not holding my breath here. We'll probably get another locked .dll.

[jochenvanwylick]

@rubenprins That is the workaround I use as well, through the Azure SDK tools in VS.

[guardrex]

There are PS commands to do it if you can get PS access to the VM. https://github.com/GuardRex/net5-iis-ps-publish/blob/7623122dbfdc55a7c53c8edd2e97443e0eea22c9/net5-iis-ps-publish.ps1#L389-L396

[guardrex]

aspnet/vsweb-publish is also very interesting; however, it looks like it uses offline-template.html, and I thought it was discovered that that method wouldn't prevent the dll locking issue ... only stopping the AppPool, deploying, and restarting the AppPool has been a stable, working approach.

we shouldn't need to muck about with custom deployment scripts when working under default conditions with VS2015

One thing has occurred to me though: Being able to use 'Publish to filesystem' and hooking a script right after the publish that has MSDeploy bits in it has enabled very simple and useful deployment story for multiple VM's. Since the script is executed sequentially, it just goes VM-to-VM right down the line. Makes a nice coffee break. :smile:

I have a question in at Get started configuring an internet facing load balancer using Azure Resource Manager with the Azure Resource Manager team asking about how to tell the load balancer to temporarily stop sending traffic to the VM while you're deploying.

Anyway ... getting back to the subject at hand that @pekkah raised. Since we know PS commands work, check for options to execute them in a VSO build:

Microsoft/vso-agent-tasks
Run a script in your build process
Running pre-build scripts in Team Foundation Service (Visual Studio Online) for Azure Continuous Deployment build process
New Build Automation Features in Visual Studio Online

... yeah ... something like that should work for you.

[brunobertechini]

Same problem here...

Still waiting for an official solution

Bruno

[moozzyk]

This will be resolved by https://github.com/aspnet/IISIntegration/issues/81.

[rubenprins]

@moozzyk, That issue will only fix deployment via dedicated tooling like msdeploy. Xcopy/File sync deployment would still not be possible, which will also block/hinder many CI scenarios that just work with ASP.NET v1-4.

[ygoe]

Issue still exists in ASP.NET Core 1.0 RC2 with Visual Studio 2015.2 MSDeploy to IIS 8 on Windows 2012 R2 with latest tools for server. Killing the dotnet.exe process or recycling the application pool resolves the issue. But that needs remote desktop access to the server and is not exactly a smooth process.

[moozzyk]

@dg9ngf - this is a bug in web.deploy now. They added app_offline.htm functionality for Azure but it's disabled by default when you deploy locally. Try opening your publish profile (PublishProfiles->*.pubxml) and change:

<EnableMSDeployAppOffline>False</EnableMSDeployAppOffline>

to

<EnableMSDeployAppOffline>True</EnableMSDeployAppOffline>

If this does not work ask @vijayrkn

[vijayrkn]

If you are using VS tooling to publish using an msdeploy profile, then this property is automatically added to your pubxml.

<EnableMSDeployAppOffline>True</EnableMSDeployAppOffline>

In your output window you should see an msdeploy command similar to this.

"C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -source:manifest='C:\Users\vramak\AppData\Local\Temp\PublishTemp\obj\SourceManifest.xml' -dest:manifest='C:\Users\vramak\AppData\Local\Temp\PublishTemp\obj\DestManifest.xml',ComputerName='https://netcoreappwithdb.scm.azurewebsites.net/msdeploy.axd',UserName='$netcoreappwithdb',Password='{PASSWORD-REMOVED-FROM-LOG}',IncludeAcls='False',AuthType='Basic' -verb:sync -enablerule:AppOffline -enableRule:DoNotDeleteRule -retryAttempts:20

-enablerule:AppOffline in the command should take care of adding the appOffline during deployment.

If custom appOffline content is needed, you can set this property in pubxml.

<AppOfflineTemplate>Path to app offline</AppOfflineTemplate>

[ygoe]

That property was _not_ present in the .pubxml file that Visual Studio created. So I added it but it didn't change anything. The -enablerule:AppOffline argument does _not_ appear in the output window.

[vijayrkn]

Can you share the msdeploy commandline displayed in the output window?

Also, can you confirm that the pubxml has WebPublishMethod as 'MSDeploy'.
<WebPublishMethod>MSDeploy</WebPublishMethod>

[ygoe]

No, the file has this instead: <WebPublishMethod>FileSystem</WebPublishMethod>

Here's the command line from the output window: Executing command ["C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -source:manifest='C:\Users\yves\AppData\Local\Temp\PublishTemp\obj\SourceManifest.xml' -dest:manifest='C:\Users\yves\AppData\Local\Temp\PublishTemp\obj\DestManifest.xml' -verb:sync -enableRule:DoNotDeleteRule -retryAttempts:20 -disablerule:BackupRule]

[bojingo]

I am deploying using the "Azure Web App Deployment" task in VSTS, as described in this blog post: http://donovanbrown.com/post/2016/05/30/DevOps-for-ASPNET-Core-RC2

Is there a way to specify the <EnableMSDeployAppOffline>True</EnableMSDeployAppOffline> using that method?

[joaomello-zz]

Hi, I had the same issue at Azure, I solved on this way https://github.com/davidebbo/WAWSDeploy/issues/14

[vijayrkn]

@dg9ngf In the current version of powershell script, we support appOffline only for WebPublishMethod - 'MSDeploy'.

AppOffline support for file system publish will be available in the next release.

[davenewza]

@bojingo I'm in the same situation. Did you figure this out?

[bojingo]

@davenewza: If you visit that blog post again, the solution is in the comments.
http://donovanbrown.com/post/2016/05/30/DevOps-for-ASPNET-Core-RC2

Basically you need to add VSTS steps to stop/start the site. Very far from an ideal solution (actually, quite lame), especially since I'm not using deployment slots, but it works okay for our dev/test environment.

There is a new preview ARM version of the web app deployment VSTS task that is msdeploy-based and promises to fix it, but I didn't have luck with it because I needed the client to create a service principal for the subscription, and that was a hassle. Maybe you can get that to work:
https://github.com/Microsoft/vsts-tasks/tree/master/Tasks/AzureRmWebAppDeployment

[tomfanning]

@vijayrkn The Publish Web dialog in VS2015 does not contain any option to set msdeploy - only gives the options for WebPublishMethod=FileSystem.

Can you provide a sample .pubxml for WebPublishMethod=msdeploy please?

[vijayrkn]

@tomfanning For an ASP.NET core console application, the only publish option currently available is file system. Are you trying to publish a console application? For a web app, all these 4 options should be available (webdeploy, web deploy package, file system and ftp).
Sample webdeploy pubxml - https://github.com/dotnetpublish/AspNetCoreBlogList/tree/master/src/AspNetCoreBlogList/Properties/PublishProfiles

If it is a web app and you are seeing only file system publish option, can you check the xproj to see if it imports this?
<Import Project="$(VSToolsPath)\DotNet.Web\Microsoft.DotNet.Web.targets" Condition="'$(VSToolsPath)' != ''" />

[bragma]

@bojingo I've been able to setup service principal, there are several tutorials out on the net. I don't have the links here but I can look for them if you need. Now my problem is that the task requires a parameters.xml file which is not generate on ASP Net Core publish... :(

[jdshkolnik]

I have TFS 2015 on-prem and this remains an incredibly annoying problem.

[michaellandi]

I'm also running into this issue...

[BobbyTable]

My solution was to keep my staging-slot that I deploy to stopped, then publish to it (auto swap works). Restarting, then deploying didn't work (deploying with VSTS)

[michaellandi]

I am using octopus with Azure Web Apps, which doesn't present an option for staging slots. However, checking the following options seems to have worked:

• Safely bring down app domain by with blank app_offline.html in root.
• Remove files on the destination that are not part of the deployment

[pholly]

I'm using Visual Studio 2015 Update 3 to publish a asp.net core 1.1 / .net framework 4.5.2 site to IIS. It drops down App_Offline.htm which does not take down the site. If I rename it on the server to app_offline.htm the site comes down.

[vijayrkn]

Per this app_offline.htm should be case insensitive.

https://github.com/aspnet/IISIntegration/issues/81

Deployer uploads app_offline.htm (case insensitive)

[pholly]

I just published again and see App_Offline.htm dropped and my site did not go down and I get the file in use error. When I change it to app_offline.htm my site goes down and I can publish with with no errors.

I am hosting on a shared UNC path - I don't know if that makes a difference.

[vijayrkn]

@Tratcher - do you know why casing of App_Offline.htm would make a difference here?

[Tratcher]

@vijayrkn see https://github.com/aspnet/AspNetCoreModule/issues/50

[BenBarreth]

I was able to solve this issue by adding an Azure appsetting of

MSDEPLOY_RENAME_LOCKED_FILES = 1

... as described here:
https://github.com/Microsoft/vsts-tasks/issues/1607#issuecomment-260946957

YMMV

[jdshkolnik]

@BenBarreth Is there something similar applicable to non-Azure?

[BenBarreth]

not that I know of sorry @jdshkolnik

[bojingo]

It seems that there may be issues with MSDEPLOY_RENAME_LOCKED_FILES = 1. It helps the deploy succeed, but may mean that the new DLLs don't get picked up at runtime:
https://github.com/Azure/azure-webjobs-sdk-script/issues/569#issuecomment-264490818

That doesn't seem like an appropriate "fix" to me. It just hides the deployment failure.

True fix may possibly be described in this issue:
https://github.com/Azure/azure-webjobs-sdk-script/issues/1023

Really looking for resolution to this. It looked like RM web deploy VSTS task took care of this with the "Take App Offline" option selected, but as of this weekend we are constantly getting the errors. We've reverted to stopping the deployment slot, deploying, and starting again.

[ctolkien]

Likewise with us, this issue cropped up on Friday last week and has blocked us from deploying since then...

[dmytro-gokun]

Same here. Looks like it only affects sites with Always On = true, but that's is not for sure.

[rsnj]

I still have this issue even when setting the MSDEPLOY_RENAME_LOCKED_FILES = 1 in my configuration. It is nearly impossible to update the build on my server now. This just started happening a few days ago.

[adonnelly]

The same issue has started occurring for me this week. Up until this week deploying an AzureRM app service in VSTS with the setting Take App Offline = true consistently worked. Now this setting seems to be either ignored or the service is not taken off line in time. I have to manually restart or stop/start the app service while running the release for it too exceed.

[urbanfarmer]

Same issue here. AzureRm was working for months and just suddenly stopped. My last deploy was two weeks ago on 12/5 and now on 12/20 it is generating the Error_File_In_Use. Very annoyed to have to go back to the manual start/stop.

[ctolkien]

Here's our workaround in Powershell that we're using for now as part of our CI/build process. Special thanks to @appveyor guys for help with this:

$username = $env:deployusername
$password = $env:deploypassword
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$Headers = @{
  'Authorization' = ('Basic {0}' -f $base64AuthInfo)
  'If-Match'      = '*'
}
$apiUrl = "https://YourAzureWebsite.scm.azurewebsites.net/api/vfs/site/wwwroot/app_offline.htm"
$filePath = "app_offline_tmp.htm"
Invoke-RestMethod -Uri $apiUrl -Headers $Headers -Method PUT -InFile $filePath -ContentType "multipart/form-data"

This makes use of the Kudu REST API to HTTP PUT the correctly cased app_offline.htm file in place. We have WebDeploy configured to remove any files not in the deployment, so it automatically removes the file as part of the deployment. You may need to repeat the step using a HTTP DELETE to remove the file after your deployment as finished if you're not using this mode.

[urbanfarmer]

Adding a Trackyon Stop before the AzureRM task and a Trackyon Start task afterwards is working well for me. This should not be necessary with the TakeAppOffline = true but it's working and it avoids manual or scripting. Just add the task to the release definition. The Trackyon steps are easy to set up

On Dec 21, 2016, at 12:10 AM, Chad T <[email protected][e-mail]> wrote:

Here's our workaround in Powershell that we're using for now as part of our CI/build process. Special thanks to @appveyorhttps://github.com/appveyor guys for help with this:

$username = $env:deployusername
$password = $env:deploypassword
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$Headers = @{
'Authorization' = ('Basic {0}' -f $base64AuthInfo)
'If-Match' = '*'
}
$apiUrl = "https://YourAzureWebsite.scm.azurewebsites.net/api/vfs/site/wwwroot/app_offline.htm"
$filePath = "app_offline_tmp.htm"
Invoke-RestMethod -Uri $apiUrl -Headers $Headers -Method PUT -InFile $filePath -ContentType "multipart/form-data"

This makes use of the Kudu REST APIhttps://github.com/projectkudu/kudu/wiki/REST-API to HTTP PUT the correctly cased app_offline.htm file in place. We have WebDeploy configured to remove any files not in the deployment, so it automatically removes the file as part of the deployment. You may need to repeat the step using a HTTP DELETE to remove the file after your deployment as finished if you're not using this mode.

-
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/aspnet/Home/issues/694#issuecomment-268436959, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AID9WoUdbuBnFoXR2K5o8AiUtv8nE9dRks5rKLTcgaJpZM4FJp_R.

---

This message is non-public and may contain privileged, proprietary, or confidential information. If you have received it in error, please notify the sender immediately and delete all copies of the original. Any other use of the e-mail by you is prohibited. Any unauthorized copying, storage, use or disclosure of the contents of this message is not permitted and may be unlawful. Where permitted by local law, electronic communications with Lithero, may be monitored and scanned by our systems for the purposes of information security and assessment of internal compliance with Lithero policies and applicable laws.

[moozzyk]

/cc @davidebbo

[davidebbo]

Try setting MSDEPLOY_RENAME_LOCKED_FILES=1 in the Azure App Settings for your app.

[ctolkien]

@davidebbo this doesn't seem to resolve things.

I assume this is related to a change on the Azure side of the fence? A lot of people (especially on the aspnet slack channel) have had this issue all crop up at the same time.

[jdshkolnik]

A change in Azure wouldn't explain why we're seeing this on-prem with TFS. Could it be the agent?

[bmoscao]

We're seeing the same problem. App_Offline.htm is not picked up. Weirdly enough if we deploy from VS it works well.

[dmytro-gokun]

Weirdly enough, for me it does not work from VS, as described here: #1883

[knuterik91]

My team also encounters this problem, almost every time deployment fails due to locked file. Needs to restart every app in azure and re-run deployment. We are using Octopus Deploy.

[mxa0079]

My team encountered this problem this morning. Any comments from the Microsoft team? The release pipeline is a critical to our productivity, this is preventing releases to production sites. Thanks!

[moozzyk]

@bmoscao - app_offline.html is case sensitive.

[dwdickens]

This problem has also appeared for me on Azure. Nothing I've done at my end to cause it.

[gilmishal]

I am having this problem too. The thing is the EnableMSDeployAppOffline is set to true - and the project output shows -enablerule:AppOffline as part of the command being ran by visual studio. I am not sure what to do... I have only started getting this issue recently..

[dtmnash]

Our workaround for this was to stop the app pool, deploy, then restart the app pool. We are using Release Manager and the inline powershell plugin to accomplish this.

Stop:

param( [string]$webApp, [string]$subscriptionName, [string]$resourcegroupName );  
Select-AzureRmSubscription -SubscriptionName $subscriptionName; 
Stop-AzureRmWebapp -Name $webApp -ResourceGroupName $resourcegroupName

Start:

param( [string]$webApp, [string]$subscriptionName, [string]$resourcegroupName );  
Select-AzureRmSubscription -SubscriptionName $subscriptionName; 
Start-AzureRmWebapp -Name $webApp -ResourceGroupName $resourcegroupName

Definately would appreciate some input from Microsoft though, this used to work great with just enabling the appOffline rule on the RM Azure deploy step.

@davidebbo We are using ASP.NET Core, VSTS Release Manager, and AzureRmWebApp with slots.

[cpunella]

@davidebbo thanks! It works for me ;)

[scottgerring]

Happening to us as well - msdeploy to Azure. Glad to see it's mysteriously started for a number of people at the same time - and it's not just us - but not super reassuring overall!

[bragma]

I don't think people at MS needs confirmation for this, probably they know well what's the problem. In any case, same here. Deploying from VSTS, stopped to work in early december. At the moment I am using trackyon task to stop/start.

[davidebbo]

Going through this thread, for each entry, it's not always clear:

1. whether issue refers to Azure App Service or some other hosting
2. whether people have tried changing the app_offline casing as discussed above. Seems several people reported success there.
3. whether MSDEPLOY_RENAME_LOCKED_FILES was tried. Again, several people seems to have been successful with that.

If you are running into this issue, please make it very explicit what your scenario is and what you have tried, so we can make sense of this. Thanks!

[fabiano]

@davidebbo

I'm receiving the message ERROR_FILE_IN_USE when I try to deploy using Visual Studio/MSBuild. The task is creating an App_Offline.htm file on the server but it's not taking the app down.

Creating an app_offline.htm file using Web Deploy takes the app down.

This is my setup:

• Server: IIS 8, .NET Core Windows Server Hosting 1.1.0, and Web Deploy 3.6
• My machine: Visual Studio 2015 Update 3 and .NET Core 1.0.1 Tools Preview 2
• Publish profile:

<?xml version="1.0" encoding="utf-8"?>

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <_SavePWD>False</_SavePWD>
    <ADUsesOwinOrOpenIdConnect>False</ADUsesOwinOrOpenIdConnect>
    <AllowUntrustedCertificate>True</AllowUntrustedCertificate>
    <DeployIisAppPath>...</DeployIisAppPath>
    <EnableMSDeployAppOffline>True</EnableMSDeployAppOffline>
    <EnableMSDeployBackup>True</EnableMSDeployBackup>
    <EnvironmentName>Production</EnvironmentName>
    <ExcludeApp_Data>False</ExcludeApp_Data>
    <LastUsedBuildConfiguration>Release</LastUsedBuildConfiguration>
    <LastUsedPlatform>Any CPU</LastUsedPlatform>
    <LaunchSiteAfterPublish>True</LaunchSiteAfterPublish>
    <MSDeployPublishMethod>WMSVC</MSDeployPublishMethod>
    <MSDeployServiceURL>...</MSDeployServiceURL>
    <PublishFramework>netcoreapp1.1</PublishFramework>
    <PublishRuntime>win81-x64</PublishRuntime>
    <RemoteSitePhysicalPath />
    <SiteUrlToLaunchAfterPublish />
    <SkipExtraFilesOnServer>False</SkipExtraFilesOnServer>
    <UsePowerShell>True</UsePowerShell>
    <UserName>...</UserName>
    <WebPublishMethod>MSDeploy</WebPublishMethod>
  </PropertyGroup>
</Project>

[pgmolloy]

@davidebbo

I have commented on this error on a similar thread and have been having to stop and restart my Azure web app every time I want to redeploy (just in development environment). Never had to do this before this week. I decided to basically create a Visual Studio 2015 .net core web app from the template without any modifications.

I ran the web app on local host no problem at all.
I publish the new app to my Azure Web App Service Plan no problem and the browser immediately brought up the site no problem.
I checked Azure Dashboard and no problems at all.

I then went and republished the web app without modifying anything - code or settings and the following error presents itself.

From then on I have to stop the web app, publish to Azure and then start the web app. This never happened to me before. This is a very simple deployment of the Microsoft template.

I recorded this whole process and sent that feedback within the Microsoft 2015 VS feedback system. Hope you can find that recording. Or I could probably do it again.

Regards Peter

[ctolkien]

@davidebbo

whether issue refers to Azure App Service or some other hosting

Azure App Service.

whether people have tried changing the app_offline casing as discussed above. Seems several people reported success there.

Changing App_Offline to app_offline fixes the issue - this file is created via webdeploy automatically with the 'App_Offline' casing as part of deployment, which is the crux of the issue.

[scottgerring]

@davidebbo
Exactly the same as @ctolkien.

[bragma]

@davidebbo

whether issue refers to Azure App Service or some other hosting

In my case, I am deploying to Azure App Service. Don't know about other hosting.

whether people have tried changing the app_offline casing as discussed above. Seems several people reported success there.

I am using the "Deploy AzureRM Web App" task from VSTS (https://aka.ms/azurermwebdeployreadme). There is an option "Publish using Web Deploy" <- checked and "Take App Offline" <- checked which used to work, until something changed somewhere and it stopped to work.
The "info" tooltip explicitly tells about placing "app_offline.htm" with lower case "a".
I am not sure how to change it using this script, it should already be lower case.

whether MSDEPLOY_RENAME_LOCKED_FILES was tried. Again, several people seems to have been successful with that.

Not tried. Don't know how to do that. But in any case, I'd prefer to take the app offline instead of changing locked files contents.

[davidebbo]

Thanks all. So it sounds clear that the primary issue is with the casing of app_offline.htm, and not anything related to Azure App Service (which is the side I come from). And this is tracked by https://github.com/aspnet/AspNetCoreModule/issues/50.

I'd suggest closing this issue and keeping that other one. I'll let the ASP.NET experts take it from there, unless there is reason to think that Azure Web App is partly to blame (unlikely based on @fabiano reporting the same outside of Azure).

[moozzyk]

Actually this looks like a regression in tooling where it seems to have started creating App_Offline.html instead of app_offline.html. @mlorbetske, @vijayrkn - any idea what it can be?

[davidebbo]

I see, so you're saying that the core runtime (AspNetCoreModule) was always case sensitive, but that it didn't use to be a problem because VS was using the matching case and now it isn't?

Though regardless, I'd argue that the runtime should not be case sensitive here, so there are two possible paths forward to address the issue.

[moozzyk]

@davidebbo - this is correct. I belive AspNetCoreModule was always case sensitive.

[ctolkien]

@davidebbo

and not anything related to Azure App Service (which is the side I come from)

I guess the reason for bringing in Azure AppService to the discussion is because without any changes on our behalf, things just started to fail, we did not change any of our package versions. Was the ANCM revision rev'd on App Services?

[davidebbo]

@ctolkien yes, I think it was indeed updated. So it could very well be that the above conclusion is not correct. Instead the new theory is:

• The older ANCM (7.1.1965.0) was not case sensitive.
• The newer one (7.1.1970.0) became case sensitive.
• Nothing changed on the VS publishing side, and the casing has always been App_Offline.htm.

@moozzyk do you think that's a possibility, or are you positive that it was always case sensitive?

[vijayrkn]

@moozzyk @davidebbo - There was no change in the VS tooling for app_offline casing. VS tooling calls msdeploy with the appoffline rule (-enablerule:AppOffline) and msdeploy drops App_Offline.htm.

This issue looks like a behavior change in ANCM. Based on the first comment here app_offline should be case in-sensitive (https://github.com/aspnet/IISIntegration/issues/81).

[oyvindvol]

@ctolkien

Changing App_Offline to app_offline fixes the issue

Where do I change this? I do MSDEPLOY directly from Visual Studio to Azure.

[ctolkien]

@oyvindvol

Where do I change this? I do MSDEPLOY directly from Visual Studio to Azure.

You can't change the casing of App_Offline from MSDEPLOY as far as I know, you'll need to custom script a solution for now. You can use the powershell script I posted above as a starting point.

[davidebbo]

To update everyone, we confirmed that the theory above is correct: the case sensitivity bug is a regression in ANCM's newer version, and that version got deployed to Azure App Service in December, causing users to suddenly hit this.

The plan is to get an ANCM fix from the ASP.NET Core team and deploy it to App Service. There is no ETA yet, but it should happen sometime in January.

[ChristopherMank]

Thanks!

[Mickael-Lamare]

@davidebbo will we b able to download ANCM to install on our own server using iis and hitting this particular bug too?

[davidebbo]

@Pictuel I would think so. Note that my angle here is strictly Azure App Service, so I'll let the Core team comment on non-Azure scenarios.

[Mickael-Lamare]

thanks for the heads up :)

[DamianEdwards]

@Pictuel yes we'll ensure that an updated version of the Windows hosting installer for .NET Core will be released that contains the update to ANCM.

@shirhatti

[guardrex]

@shirhatti I'm monitoring here when the updated installer is available to update doc links.

[sabbour]

After going through that thread I'm not really sure what the workaround is exactly for App Services. Is it the App Setting or is it using a custom deploy script or stopping and starting the site?

If it is the app setting I've read another comment saying it causes the website not to pickup the new DLLs.

[Mickael-Lamare]

If I understood correctly there is no workaround on our side until an update is available on ANCM (https://github.com/aspnet/AspNetCoreModule/issues/50). App services should be updated by the azure team.

[davidebbo]

I think you're best bet on App Service is to stop/start the app as described by @dtmnash here.

[pksorensen]

@davidebbo "The plan is to get an ANCM fix from the ASP.NET Core team and deploy it to App Service. There is no ETA yet, but it should happen sometime in January." -- Let us know when an ETA is ready on this :)

[davidebbo]

Tentative deployment ETA is between 24th and 30th. Deployment happens gradually over the many scale units, so not everyone will get the fix at the exact same time.

[hbunjes]

Thank you, for me it just started working :-)

[rsnj]

Still an issue for me :(

[davidebbo]

@hbunjes thanks for confirming!

@rsnj it's happening progressively over some period of time. It should all be done by end of month (unless we hit an issue).

[henningst]

@davidebbo Deployment seems to work for some of my apps but not all. The weird thing is that some deployments fail but others don't for apps running in the same App Service Plan. Is it possible to see what version of ANCM is running for a particular web app?

[davidebbo]

@henningst See https://github.com/aspnet/AspNetCoreModule/issues/50#issuecomment-271381892 for a way to check.

[BenBarreth]

Was not working for me yesterday but it IS now. Thank you @davidebbo!

[clement911]

It started working for me too. Thanks!

[davidebbo]

Yes, the deployment is now complete, so it should work for everyone.

[ctolkien]

All good for me here as well.

[pekkah]

Seems to be working now. From report it took over one year to get this fixed. I would've though deployment issues get higher priority :p

[jdshkolnik]

How to fix this for non-Azure?

[davidebbo]

@pekkah I think it went through distinct issues. The latest one was much more recent than one year.

@jdshkolnik for non-Azure, I guess you just need to start using the new ANCM.

[shirhatti]

@jdshkolnik
The update has been available for download at https://www.microsoft.com/net/download/core#/runtime
This is a direct link to the download.

[jdshkolnik]

@shirhatti I installed this yet am still getting these errors.

[rpicone]

I´m still getting the error in Sao Paulo, South Brazil.

[shirhatti]

@jdshkolnik What version do you see when you run this in PowerShell?

[System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Windows\System32\inetsrv\aspnetcore.dll").FileVersion

[jdshkolnik]

@shirhatti 7.1.1971.0

[ajbeaven]

Update: sorry, this was my fault. I was reviewing and updating an existing Release's deployment steps rather than the release definition. I've lost count of how many times I've done this...

I'm still coming across this error every time I try to deploy even though my version of aspnetcore.dll is the one that is supposedly fixed.

aspnetcore.dll version: 7.1.1971.0
Deployed using: VSTS task Azure App Service Deploy v3.* (preview)
Take App Offline: Checked
Remove additional files at destination: Checked
Rename locked files: Checked
MSDEPLOY_RENAME_LOCKED_FILES app setting: Not added

At this stage it's not clear to me what settings are or are not required for this to work. Could someone provide any insight here?

[pksorensen]

Just had my first file in use issue again now on azure app services:

2017-02-14T22:41:10.4400186Z C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe -source:IisApp='D:/a/r1/a/Ascend Ammo Portal/drop/apps/artifacts/AmmoPortal' -dest:IisApp='core-asyoz77ajoed4/ammo-preview',ComputerName='https://core-asyoz77ajoed4.scm.azurewebsites.net/msdeploy.axd?site=core-asyoz77ajoed4/ammo-preview',UserName='$core-asyoz77ajoed4',Password='**',IncludeAcls='False',AuthType='Basic' -verb:sync -retryAttempts=20 -verbose -enablerule:AppOffline

2017-02-14T22:41:34.3837386Z Error Code: ERROR_FILE_IN_USE

2017-02-14T22:41:34.3837386Z More Information: Web Deploy cannot modify the file 'Ascend.Ammo.Portal.exe' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.

2017-02-14T22:41:34.3837386Z Error count: 1.

[dfaivre]

FYI: the docs seem to link to an installer with the old version (*.1970): https://docs.microsoft.com/en-us/aspnet/core/publishing/iis#install-the-net-core-windows-server-hosting-bundle.

Also, seems like on the runtime downloads page only the LTS version has the fix?

[guardrex]

@shirhatti ~Let me know when the new hosting bundle installer link is ready, and I'll shoot it into the docs (it's in a few places).~

~Is it this one: https://go.microsoft.com/fwlink/?linkid=837808 ... that's not an aka.ms link tho.~

I setup https://github.com/aspnet/Docs/pull/2773 to cover this if the fwlink is ok.

[pksorensen]

Also happens in visual studio, and as the options show - the app offline rule is present.

[pekkah]

@davidebbo could you confirm is this fixed? Hearing from quite a many people that they're still hitting it. Did the release version of the tooling have any fix for this or is it still a hosting problem?

[pksorensen]

I am also trying to figure out why and when it happens. I am almost sure I have seen that the process is running in process explorer on scm site and it published just fine.

I am in the process of updating my sites to newest tooling also, hoping that the issue is related to old project.json tooling.

[davidebbo]

The fix was deployed to Azure App Service a while back, and I'm not aware of anyone still seeing the issue afterwards. Of course, users on non-Azure hosts may still see it if they have not been updated.

For Azure, if you are having issue, please test it as follows:

• Go to Kudu process explorer and make sure your app's process is running (typically dotnet.exe)
• Manually create an app_offline.htm file (e.g. run touch app_offline.htm) from Kudu command
• Check process explorer again, and make sure that process is no longer there.

[pksorensen]

I will test some more, did you see the image a few posts up, it clearly shows that file is in use and it uses app offline.

But i will test as you asked.

Also, we are using scaleout + apps deployed to virtual applications - dont know if that changes anything.

[davidebbo]

@pksorensen doing it 'manually' will help isolate from anything WebDeploy might be doing. i.e. if simply dropping app_offline.htm doesn't stop the process, then we really know there is an issue not caused by WebDeploy.

Note that normally, you end up with a dotnet.exe process, while in your case you have a different exe name. I wonder if that somehow relates to the issue.

[pksorensen]

1. Go to Kudu process explorer and make sure your app's process is running (typically dotnet.exe)
   I did this, but its not dotnet.exe but our name compiled bin Ascend.Ammo.RegistrationTablet.exe

Then i did touch app_offline.htm, but this did not kill the process.

Am i doing something wrong since I dont see dotnet.exe?

[davidebbo]

I'll let the dotnet / ANCM experts comment on that part. I know by default, when deploying a core ASP.NET app, it uses dotnet.exe. I think the case when it doesn't is when you use standalone deploy (or whatever it's called). But I have no idea how that affects ANCM and app_offline.htm.

@DamianEdwards who can best comment on this?

[DarqueWarrior]

I avoid that issue using deployment slots. Works 100% of the time. Stop the slot before you copy files. This will remove everything out of memory so your copy will work no locked files. Then start the slot and swap into production. Added benefit your customers never see the app offline page. They get a 0 downtime deployment.

http://donovanbrown.com/post/MicrosoftCodeAnalysisCSharpdll-Locked-Problem-Fix

[davidebbo]

@DarqueWarrior slot works of course (and are good to use for other reasons), but the key thing in this thread is to understand why app_offline appears ineffective at shutting down the Core process for some people like @pksorensen. And I'd like the Core team to comment on that.

[shirhatti]

@davidebbo To answer your earlier question, portable vs standalone has no bearing on the app_offline behavior. The app offline file is monitored by w3wp.exe (by the ASP.NET Core module).

[pksorensen]

@DarqueWarrior Thanks for the advice, one comment though - if you as us are deploying multiply sites to virtual applications in the same app service, the slots (if i recall correct) work across all those, so you would have to deploy all sites before swapping. This is a hassle. Right now we can deploy parts (micro services) to individual virtual applications easy. Ofcause we can go about spreading those into seperate app services, but then we would need loadbalancers/gateways infront to make them share the same domain again, also costs more time then we are willing to do right now.

So I agree that what your suggesting is the preferable way, it just dont fit with us right now. So still hoping for a resolution on the File In Use issue.

[davidebbo]

@pksorensen any chance you can set up a repro in a dummy site and share its name? Basically, I'd like to see it in the state where your site is running, and manually creating app_offline.htm in Kudu console does not cause it to shut down.

@shirhatti any idea what could cause ANCM to not shut down the process? How aggressively does it try to do that?

[pksorensen]

@davidebbo It will take me a bit of time but I will try to work something out to crete a repro.

[Jon12345]

@davidebbo I created a new ASP.NET Core app, and a new Azure App Service. I'm using Azure's "Continuous Delivery (Preview)" feature, and I'm getting this issue.

2017-03-30T02:54:36.5648892Z ##[error]Failed to deploy App Service.
2017-03-30T02:54:36.5658893Z ##[error]Error Code: ERROR_FILE_IN_USE
2017-03-30T02:54:37.1850861Z More Information: Web Deploy cannot modify the file 'myApp.dll' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.
2017-03-30T02:54:37.1860517Z Error count: 1.
2017-03-30T02:54:37.4179909Z ##[error]Error: C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe failed with return code: 4294967295

Manually creating an app_offline.htm in D:\home\site\wwwroot does indeed stop the dotnet.exe process.

So does this mean there's an issue with "Continuous Delivery (Preview)" itself?

[Jon12345]

Further to the above, I fixed this by manually going to VSTS Release Definition, Additional Deployment Options, ticking "Take App Offline".

But this raises a few questions:
Why is "Take App Offline" unticked by default? Should this scenario still work (eg does it reduce downtime)?

If yes, then why is it failing? If no, then why does "Continous Delivery (Preview)" leave it unticked?

Either way - seems like there's a bug somewhere.

[davidebbo]

@Jon12345: this thread is only about discussing whether app_offline.htm is working correctly with Core apps. So that fact the VSTS may or may not be doing this by default is really a separate discussion, which should be had on a VSTS forum (probably https://social.msdn.microsoft.com/Forums/vstudio/en-US/home?forum=TFService).

At this point, most people appear to have been unblocked after the fix. There has been one report of it not working by @pksorensen, and we're waiting for a repro app to look at.

[jdshkolnik]

I still get this on TFS 2017 Update 1.

[davidebbo]

@jdshkolnik Are you sure it's set up to create the appoffline.htm file? If not, it's outside the scope of this issue. Please do the manual appoffline.htm test as outlined above.

[jdshkolnik]

@davidebbo Yes, it is. It's often the case that the scheduled nightly deployment fails its first attempt and requires a manual re-deploy attempt which usually succeeds.

FYI, my firm is federated with yours so I can easily show you via Skype for Business desktop sharing.

[davidebbo]

@jdshkolnik please make sure you do the manual test discussed above, as this helps getting everything else out of the equation.

Important: This issue is not about investigating any issue with VSTS or other publishing workflows. It is about only one thing: making sure that appoffline.htm is working.

[jdshkolnik]

@davidebbo It goes offline as soon as I manually place app_offline.htm into the folder.

I don't know how to better distinguish whether I'm perfectly on topic here. All I know is that it suddenly cropped up last year for sites that were working without issue and has regularly reared its head ever since.

##[section]Starting: Deploy IIS App: \\usserver\Deploy\CD_146_QA\Installer\Install\Web\App.Web.zip
==============================================================================
Task         : WinRM - IIS Web App Deployment
Description  : Connect via WinRM, to deploy Web project locally on IIS, using Web Deploy
Version      : 1.4.3
Author       : Microsoft Corporation
Help         : [More Information](http://aka.ms/IISWebDeploy)
==============================================================================

Preparing task execution handler.

Executing the powershell script: I:\Agent\_work\_tasks\IISWebAppDeploy_50acc50f-7d15-470b-83c1-578b3f3eeba2\1.4.3\Main.ps1
name='db-Web.config Connection String',value='Data Source=dbserver;Database=Internal_QA;Type System Version=SQL Server 2012;User ID=xxx;Password=yyy;Persist Security Info=True')

Starting deployment of IIS Web Deploy Package : \\usserver\Deploy\CD_146_QA\Installer\Install\Web\App.Web.zip

Performing deployment in sequentially on all machines.

Deployment started for machine: usserver with port 5985.

Deployment status for machine usserver : Failed

##[error]Microsoft.PowerShell.Commands.WriteErrorException: System.Exception: Error Code: ERROR_FILE_IN_USE

For more info please refer to http://aka.ms/iisextnreadme

##[error]PowerShell script completed with 1 errors.

##[section]Finishing: Deploy IIS App: \\usserver\Deploy\CD_146_QA\Installer\Install\Web\App.Web.zip

[davidebbo]

@jdshkolnik My best guess is that whatever this deployment script is doing is not creating the appoffline.htm before publishing the files. It could be a VSTS configuration issue or bug. But from the point of view of ANCM, the test that you did shows that things are working as expected when appoffline is created.

[jdshkolnik]

@davidebbo I don't know that what I did was a definitive test. After all, the second manual attempt after we get an error is usually successful. The deployment process which might block app_offline.htm wasn't running.

[alexgritton]

I'm still running into this case sensitive issue on 7.1.1971.0. When I try to deploy using VS I get a file locked error, so I looked to see if the app was still running and sure enough it was. I opened the Kudu command line to see what files where in the wwwroot directory and App_Offline.htm was in the folder but the process was still running. Then I typed touch app_offline.htm and the process stopped running as expected. I double checked the modules and my aspnetcore.dll version is 7.1.1971.0.

[davidebbo]

@alexgritton that's odd. Would seem to imply that ANCM did not initially detect the file. Does that happen consistently or randomly?

[alexgritton]

I just created the application today on Azure App Services and it's been happening all day, so I would say consistently. Do you have any other ideas of what could be causing this?

[davidebbo]

Not really. I'll let the ANCM experts comment on how they do file change monitor, and if they can think of a reason why initial creation is not detected.

[derekgreer]

@davidebbo I'm having the exact same issue. My team has a Fake build which copies a file AppBuild.htm from the build server out to the target site as app_offline.htm. We then attempt to use msdeploy to sync up the folder and get the opened by another process error. If I touch the file, rename the file from app_offline.htm to app_offline.htm.rename and back again, or overwrite app_offline.htm using File Explorer then the process stops properly. It doesn't quite make sense why it doesn't work through our script but does if we do it manually.

[derekgreer]

Update: We found that updating the last write time seems to consistently trigger the shut down:

Target "StopApi" (fun _ ->                                    
    trace "Deployment - Stopping Api"                 

    for folder in apiWebDeployShare do                        
        let offlineFile = folder @@ "app_offline.htm"         
        let offlineFileSource = folder @@ "AppOffline.htm"    
        CopyFile offlineFile offlineFileSource                
        File.SetLastWriteTime (offlineFile, DateTime.Now)     
)                

There definitely seems to be something up with the code that monitors the app_offline.htm file.

[davidebbo]

@derekgreer So what you're seeing is that when you copy the file over, the shutdown is not triggered? I'm not able to repro this from Kudu console. e.g.

• Go to Kudu Console
• Run touch app_offline.htm in D:\home\site to create an empty file in a different folder
• Run copy app_offline.htm wwwroot. Note that this preserves the last write time and updates the creation time (standard file copy behavior).

Result: the dotnet.exe process shuts down instantly (you can check using Kudu process explorer).

Do these same steps work for you? If so, what might be different in your repro scenario?

[derekgreer]

Actually, I should have also added that this issue has been intermittent and that we are using IIS 8 on Windows Server 2012 .

My intent this morning was to see if I could reproduce the issue with a powershell or batch script before setting up the Kudu Console, but having first tested with the test Fake build script my team was using on Friday to create the app_offline.htm file in various ways (copy existing file, create new file, etc.), it's currently working. This reflects the pattern of our build failures as well where it sometimes was shutting down the process and other times would not. When it does start failing, it seems to do so consistently. We tested for hours on Friday and it consistently would not stop with a simple Fake build that did absolutely nothing except create a new app_offline.htm file. I should also add that we would periodically create the file manually and see the process stop, so it wouldn't seem to be any sort of situation where the process wouldn't shut down after some extended period of usage.

When I can reproduce the issue again I'll see if creating the file using powershell and/or batch file still reflects the issue.

[derekgreer]

@davidebbo Well, crap ... I just noticed that I forgot to comment out a line that was updating the last update time, so it is indeed still showing the same behavior. I'll try with powershell and a batch file now and update you.

[derekgreer]

@davidebbo

Sorry for the confusion. Okay, here's what I've found:

Using a batch file with the following contents seems to reliably cause the process to shut down:

echo "" > \\testserver\e$\Site.Api\app_offline.htm

Using an equivalent bash script also reliably causes the process to shut down:

#!/bin/bash

echo "" > //testserver/e$/Site.Api/app_offline.htm

Using a powershell script with the following contents seems to never cause the process to shut down:

New-Item \\testserver\e$\Site.Api\app_offline.htm -type file

Given that both Fake and Powershell run under the CLR while the DOS copy command, bash, and the Kudu Console do not, this seems to point to something related to the file being created through the CLR.

The weird thing is, I see no differences in the resulting file's create/update timestamps between the various versions.

[davidebbo]

@derekgreer Oh, so you're not running on Azure App Service at all. That's my angle in this whole saga, so I'll let the ASP.NET folks comment further on this.

Though for what it's worth I was not able to repro in App Service:

• Use Kudu's PowerShell console
• Go to site\wwwroot folder
• Run New-Item app_offline.htm -type file

@moozzyk can you help investigate @derekgreer's repro outside app service?

[derekgreer]

Further info:

Since creating, copying, renaming, etc. through File Explorer has consistently worked, I decided to see if a .Net-based file explorer app reflected the same results as I'm seeing with Fake and PowerShell. There's a .Net-based file explorer named "Nomad" which I ended up downloading and testing with and the results were consistent with Fake and PowerShell. Upon creating an app_offline.htm file, the process did not stop. I then renamed the file and renamed it back to app_offline.htm again and the Core process then shut down.

[derekgreer]

https://github.com/aspnet/AspNetCoreModule/blob/93ace1b84bd79382233cb39b858611d2db05552e/src/AspNetCore/Src/filewatcher.cxx#L321

I would think this method would want to include the flag "FILE_NOTIFY_CHANGE_CREATION".

I wonder if the .Net API common to Fake, PowerShell, and Nomad doesn't trigger these other flags in the same way as normal file creation.

[davidebbo]

@shirhatti can you please make sure someone investigates this potential ANCM issue?

[shirhatti]

@davidebbo Ack. We are looking into it

[gulbanana]

I am having a very similar issue. If I copy in an app_offline.htm file in Windows Explorer then it correctly stops the process, but if I create it using COPY app_offline.htm \server\share\siteapp_offline.htm from my build server then it does not stop and files remain locked.

[jack4it]

@gulbanana I have the exactly same issue as you do. Have you solved it already? @shirhatti Any updates on your investigation? This one drives the team crazy because the CI pipeline fails all the time due to file copy failures which in turn caused by dotnet.exe not being shutdown by app_offline.htm.

[gulbanana]

I worked around the issue by using something along the lines of echo "<html>page content</html>" > \\server\share\app_offline.htm. I think @derekgreer is correct that the bug is about ANCM watching for only some methods of file creation, and it catches this one.

[jhkimnew]

@derekgreer
I am looking into issue now. With one of your repro commands in my test machine, I was able to reproduce this problem. I used below command. (NOTE: below command will create empty (file size zero) file.

new-item \\iisdist\privates\jhkim\temp\PublishOutput\app_offline.htm -ItemType file

One interesting thing is if I add some file content when creating the app_offline.htm file, there is no problem. That can be done with simply adding -value "test" parameter as the following:

new-item \\iisdist\privates\jhkim\temp\PublishOutput\app_offline.htm -ItemType file -value "test"

Can you confirm if you see the same symptom on your repro environment?
If so, I think that this issue only happens when empty app_offline.htm is created with some special way such as using the new-item powershell command.

@gulbanana
I can't reproduce this issue with Copy command. Can you explain more detail thing of your repro? It would be great if you can find some repro steps with which I can reproduce the same problem on my test machine.

[derekgreer]

I can confirm it isn't related to whether the app_offline.htm file is empty without running those commands as our experience has been with copying a file with the desired contents to the deploy folder. Based on my experience, I'd guess that the second command might be causing a file changed event if it is indeed working. Perhaps it creates the file and subsequently updates it with contents. Regardless, every .net-related process I've used to just create the file with or without contents fails to trigger the process shutdown.

@shirhatti Any update? Were you able to confirm your code should be using the "FILE_NOTIFY_CHANGE_CREATION" flag?

[shirhatti]

@derekgreer We are listening of every valid flag except change last access and change attributes.
FILE_NOTIFY_VALID_MASK & ~FILE_NOTIFY_CHANGE_LAST_ACCESS & ~FILE_NOTIFY_CHANGE_ATTRIBUTES

So yes, we are already listening for FILE_NOTIFY_CHANGE_CREATION.

cc: @pan-wang

[derekgreer]

@shirhatti Ah, I understand now. I wasn't aware that flag covered change creation. I'm curious, have you been able to reproduce the issue?

[moozzyk]

Not reacting to empty app_offline.htm was supposedly fixed: https://github.com/aspnet/IISIntegration/issues/174

[jhkimnew]

@moozzyk @derekgreer Thanks for the confirmation. Okay, I also found that the issue is not directly related to the empty file content. I will update you after figuring out the root cause.

[gulbanana]

@jhkimnew i don't have the exact broken setup anymore, but i can describe the specifics:

• IIS running on windows server 2008 r2, hosting an asp.net core application using ANCM
• build server is also 2008r2, on the same domain
• IIS server has its website physical directory shared using windows sharing
• domain user (the build server's account) runs a powershell script on build server
  if this script uses the windows copy command to copy in an app_offline from an existing file in another directory, then the hosting process does not shut down. if the script instead uses "echo" (which may be the /bin/echo from a mingw install, i'm not sure) then the hosting process does shut down.

i suspect the problem is to do with precisely which file attributes are or aren't updated by that kind of remote copy

[derekgreer]

@gulbanana Just to be clear, you tried with the dos copy command and not the power shell copy-item commandlet? I don't think I tried the regular dos copy, but I found that any .net based app I used did not work.

[gulbanana]

unless powershell has an alias for that, it was dos copy

[jack4it]

In my case, the PowerShell copy won't shutdown the dotnet.exe process, but the echo, which is an alias of Out-Content does work. I feel it's unlikely a .NET issue, but more regarding why a network copy doesn't work

[jhkimnew]

I investigated if there is any problem in the aspnetcore.dll (ANCM) change notification, but I was unable to reproduce the issue when I tried to drop app_offline.htm manually either using powershell cmdlet or DOS command. And I found one missing point which was not discussed here.
That is about the graceful shutdown. Unlike the HttpPlatformHandler", ANCM supports graceful shutdown, which means when app_offline.htm is dropped on the root directory, ANCM sends a Crtrl-C/Break signal to the backend process and waits until 10 second allowing the graceful shut down happens.
The 10 seconds is the default value and users can adjust the value with the shutdownTimeLimit of ANCM configuration settings. If the backend process is not shutdown in the configured shutdownTimeLimit, ANCM is supposed to kill the backend process.

While checking how MSDeploy publishing is related to the ANCM graceful shutdown, I finally found one consistent repro step, which is to increasing shutdown time (5 second) and noticed MSDeploy retries only 2 times by default and waiting only 2 seconds and eventually fails to publish because it does not wait the graceful shutdown time.

@vijayrkn
After I put 5 seconds delay to both place of starting and ending the aspnetcore web application, I was able to reproduce the same error, which was reported in this issue, in the second trial of publishing the aspnetcore web application to local IIS server. I attached the publish profile I used below as well.
Would you explain why this happens and give a correct guideline about how to fix this publishing problem? And please make a new issue on this separately so that your team track this issue to enhance user experience.

... <Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <PropertyGroup> <EnableMSDeployAppOffline>True</EnableMSDeployAppOffline> <WebPublishMethod>MSDeploy</WebPublishMethod> ...

[vijayrkn]

MSDeploy tries to deploy right after the app_offline is dropped. If there is a delay in stopping the process, then the deployment could fail.

Retry count for the deployment can be set using this msbuild property (https://github.com/aspnet/websdk/blob/dev/src/Publish/Microsoft.NET.Sdk.Publish.Targets/netstandard1.0/PublishTargets/Microsoft.NET.Sdk.Publish.MSDeploy.targets#L67)

for e.g, setting this property to 10 will make sure that msdeploy retries for 10 times with a one sec delay in between.
<RetryAttemptsForDeployment>10</RetryAttemptsForDeployment>

[jhkimnew]

@vijayrkn

Thanks for the information. After adding the two lines below, I don't see any problem with the same repro environment.
<EnableMSDeployAppOffline>True</EnableMSDeployAppOffline> <RetryAttemptsForDeployment>10</RetryAttemptsForDeployment>

So, anyone who runs into the similar issue would need to increase the RetryAttemptsForDeployment count with appropriate value and try again if that can solve the problem.

NOTE: The reason I added EnableMSDeployAppOffline is because that is disabled by default when using local IIS server as the destination of publishing an aspnetcore web application. So, if you are not sure whether that is enabled or not in your environment, you will just need to add the line as well and that's why I added it here for your information.

[derekgreer]

Just to ensure the original issue doesn't get lost due to the most recent discussions, the issue my team experienced is most definitely not a timing issue. We've tested manually creating the app_offline.htm file through both Powershell, Fake, and a .Net-based file explorer named "Nomad" and the process doesn't shut down no matter how long you wait. Creating the file through File Explorer, a DOS batch file, or a bash script (i.e. non .Net library-based processes) causes the process to shut down within 1-2 seconds.

This would seem to indicate:

1. The problem isn't network related (all tests were performed against a remote file share)
2. It isn't a graceful shutdown/timing issue

The fact that 3 non-.Net methods of creating the app_offline.htm file shutdown the process reliably and quickly and that 3 .Net-library based methods of creating the file never result in the process shutting down in our tests seems to be a really, really big indicator to me.

[jhkimnew]

@derekgreer
I can't reproduce the issue even with "Nomad".
Here is what I did on my Windows 10 (amd64) machine.

1. Install IIS
2. Install Visual Studio 2017
3. Create a AspnetCore web application and deploy to the local IIS server with MSDeploy
4. Install Nomad v3.2.0.2890 (http://www.nomad-net.info/downloads)
5. Send a request to the web application
6. Open Task Manager and confirm dotnet.exe process is running
7. Open Nomad and create a new file named app_offline.htm on the root directory of the web application
8. See the Task Manager and verify the dotnet.exe process is gone when the app_offline.htm file is created

Would you give the exact repro steps so that I can follow to reproduce the issue?

[derekgreer]

Aside from the fact that our .Net Core application is actually a standard .Net 4.5.2 Console Application created in VS 2015 which references ASP.Net Core assemblies, I see no difference. The project is about a year old now and was set up that way at the time due to limitations with referencing core template project types from .Net framework template project types and various sorts of testing library support for .Net core.

[pan-wang]

We identified couple issues: 1) dropping app_offline.htm file (using some tool) sometimes only triggered file property change notification which was filtered out by ANCM. 2) ANCM might fail to read app_offline.htm as the latter was held by the file copying tool. 3) graceful shutdown caused delay in terminating the backend process.
We will address the first two issues in next release. For graceful shutdown case, user needs to retry.

[gulbanana]

sounds good. my case was definitely not about a graceful shutdown timeout - when it does stop it stops immediately.

[ChuckkNorris]

Any update on this issue?
Getting while trying to deploy an ASP.NET Core 1.1 app to an Azure Web App slot using VSTS

• I have the MSDEPLOY_RENAME_LOCKED_FILES=1 set in the Application Settings
• Manage App Service Task is configured to take the app offline
• Deploy App Service task _was_ configured to take the app offline, but that didn't work either

The core issue is that the DLL is in use:

[davidebbo]

@ChuckkNorris please see https://github.com/Microsoft/vsts-tasks/issues/1607 for latest on this.

[ChuckkNorris]

@davidebbo Thanks for your reply, David. I did see that thread and that's where I discovered many of the workarounds I tried.

As this issue is still open and the error is still prevalent, I was hoping there might be a fix soon, not another workaround. This issue has been open for over two years now after all.

[davidebbo]

@ChuckkNorris while the symptom is the same, this is not the same issue:

• Original issue related to casing of app_offline.htm, causing it to be ignored completely. That has been fixed for a while.
• This new issue started just a week or so ago, and happens because now Core takes longer to shut down when app_offline.htm is created, and msdeploy by defaulty doesn't wait long enough. The workaround is to increase the retry count.

[DaveSlinn]

I just updated VS 2017 to 15.3 and install .NET core 2.0 SDK. Created a simple MVC app in VS - it was targeting netcoreapp1.1. I published to Azure, and everything was fine. I upgraded to newcoreapp2.0 and then upgraded all the nuget packages to 2.0 (Microsoft.AspNetCore, Microsoft.AspNetCore.Mvc, etc)

When attempting to publish to Azure (i.e. overwrite the 1.1), I'm getting this

Error : Web deployment task failed. (Web Deploy cannot modify the file 'Microsoft.ApplicationInsights.AspNetCore.dll' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.)

[davidebbo]

@DaveSlinn please see https://github.com/Azure/app-service-announcements/issues/24

[ctolkien]

I have had to disable MvcViewComplication to get my 2.0 apps to deploy via git and prevent file in use issues.

[vincent1173]

https://github.com/Microsoft/vsts-tasks/issues/5259
https://github.com/Microsoft/vsts-tasks/issues/1607#issuecomment-349115532
@davidebbo , Can we get a closure on this issue?
Despite adding a retry helper for web deploy, this issue is persisting.

[jeremycook]

Just wanted to point out the obvious, that this affects FTP publishing as well.

[davidebbo]

@jeremycook FTP is a bit different, as it has no automation for using App_Offline. You'd need to manually create that file to get the process to shut down, after which you could copy files.

[jeremycook]

@davidebbo fair enough but it seems like if I publish via FTP using Visual Studio's publish feature that it should just work. Perhaps I'm posting to the wrong place?

[davidebbo]

@jeremycook frankly, I'm not sure what VS does when doing FTP deployment. Maybe others can shine in, but it seems more like a VS question than ASP.NET.

Though to isolate, it would be interesting to test whether deployment is successful if you manually create App_Offline.html, and then FTP publish from VS.

[shirhatti]

@vijayrkn Can you comment on VS behavior during FTP deployment?

[vijayrkn]

FTP publish from VS to App service does not support App_Offline.

[davidebbo]

@vijayrkn ok, so for all practical purpose, we can say that .NET Core deployment via FTP is not supported, unless user explicitly stops the site before deployment. Right?

[vijayrkn]

@davidebbo - Yes, that is correct.

[pekkah]

Hitting this issue when deploying asp.net core 2.0 web app to IIS from VSTS release using the IIS web deploy template. Delete fails due to file in use. App offline option is enabled.

[jeremycook]

@davidebbo @vijayrkn @shirhatti this is all a bit depressing. Are there plans to get FTP/XCOPY working? Should a new issue be opened that focuses on deploying the old fashioned ways via FTP/XCOPY/ROBOCOPY/etc.?

[figuerres]

i am late to this party but here is a few bits i want to report:
VS 2017 Enterprise
asp.net 4.6.xx
server is IIS 8.5 on server 2012 R2
sql server types dlls are locked and the lock is such when a different developer does a publish with msdeploy they get an error and the web site is now broken as only part of the deploy ran.

the sql types dlls are locked in the app bin folder and only them.
rename the dll files and then i can publish.
the sql types packaging has caned code for loading the unmanaged dlls.
seems like that should be changed to not lock them. if that can be done.

i have seen folks talking about ways to re-start the site or take it offline but i have not found a setting in msdeploy for it. the xml i have seen is not in my files.

better guidance should be added to the sql types package that microsoft publishes.

[buzallen]

Posting on this issue as it seems more directly related to mine. I've also followed the below issue on the same topic though it is for VSTS. I am publishing directly from visual studio.

https://github.com/Microsoft/vsts-tasks/issues/5259#issuecomment-350940342

This has been an on and off issue for me but recently returned in force and on one of my app service sites I get this error on almost all of the publish attempts. This site is hosted on a 'basic' level service so I don't have slots that I use on other apps to get around this issue.

This is for a .net core 2 mvc application.

[figuerres]

One thought : some files that are a part of a web app are not chnaging on every publish (like the sql types)
but the ms deploy is trying to replace them on every deploy.
seems like some kind of check should happen to no re-copy a file that has not changed and that would make the update smaller and faster and less need to mess with the file locking.
multiple benefits if that can be done.

the other thing is to better identify what is locking the file and why and how to undo that.
and get that as a part of the standard deployment pipeline.

[stijnherreman]

@figuerres I've been wanting to deal with that issue for a couple of months now. Locking issues are normally avoided due to Shadow Copying, but by loading those native DLLs from ~/bin, a locking issue is created. I've a potential solution, but I'm not sure if it's a completely safe solution: https://stackoverflow.com/questions/47796553/is-it-safe-to-manually-copy-native-dlls-to-a-shadow-copy-directory

I agree that the documentation for the NuGet package should provide a better method, one that does not cause locking issues.

[stijnherreman]

@figuerres FYI, I've implemented the above, see the Stack Overflow question for the code.

[Dishanx]

Kill dotnet.exe in Task Manager

[oliverjanik]

What's the status here? 3 years on, still not fixed. I have to manually go to the server and stop the website.

[eplowe]

3 years? The issue was reported August of 2017. Off by 1 like whoa.
On Mon, Apr 23, 2018 at 8:56 PM Oliver Janik [e-mail]
wrote:

What's the status here? 3 years on, still not fixed. I have to manually go
to the server and stop the website.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/aspnet/Home/issues/694#issuecomment-383768450, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAXhfrhOTvZn2qb4kU-Y1Kg9I-IQKD9yks5trnhXgaJpZM4FJp_R
.

[oliverjanik]

The very first post says " pekkah opened this issue on 23 Jun 2015"

[eplowe]

You’re right. My fault. I only saw the last timestamp in the email chain.
On Mon, Apr 23, 2018 at 9:06 PM Oliver Janik [e-mail]
wrote:

The very first post says "pekkah commented on 23 Jun 2015"

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/aspnet/Home/issues/694#issuecomment-383769982, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAXhfgwYG-jLjnoJF7MCMtzbbfxSZkt2ks5trnqXgaJpZM4FJp_R
.

[Eilon]

Closing because this is as old as dirt 😄

[fubar-coder]

@Eilon Sorry, but why was this closed? It's still unresolved?

[bartdk-be]

Closing because this is as old as dirt 😄

But it's still an issue .. ? (even on none-azure deploys)

[fubar-coder]

Created a new issue (#3719) to increase the visibility.