Aspnetcore: Deploying outofprocess app: HTTP Error 502.5 - ANCM Out-Of-Process Startup Failure

[fierro]

Describe the bug

I cannot get this web app to run out off Azure windows app service. i had other tests apps up but this one is evading me. Locally I run in process, but my research says I would have better luck on azure doing out of process so i am attempting that.

The messages I get:

HTTP Error 502.5 - ANCM Out-Of-Process Startup Failure
This error occurs when a CGI application does not return a valid set of HTTP headers, 
or when a proxy or gateway was unable to send the request to a parent gateway. You may need to get a network trace or contact the proxy server administrator, if it is not a CGI problem.

To Reproduce

Steps to reproduce the behavior:

1. ASP.NET Core '2.2'

Expected behavior

It would simply run

Additional context

web.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath=".\Sabio.Web.Api.exe" stdoutLogEnabled="true" stdoutLogFile="\\?\%home%\LogFiles\stdout" hostingModel="outofprocess">
      <handlerSettings>
        <handlerSetting name="debugLevel" value="FILE,CONSOLE,TRACE,INFORMATION" />
        <handlerSetting name="debugFile" value="aspnetcore-debug.log" />
      </handlerSettings>
      <environmentVariables>
        <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Development" />
        <environmentVariable name="ASPNETCORE_HTTPS_PORT" value="44373" />
      </environmentVariables>
    </aspNetCore>
    <security></security>
  </system.webServer>
</configuration>

What i think are the meaningful parts of logs

aspnetcorev2_outofprocess.dll] Starting file watcher thread
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:730 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' , ErrorCode = '0x8027025b', processId '12852', processStatus '-532462766'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:894 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' at stage 'PostStartCheck', ErrorCode = '0x8027025b', assigned port 15507, retryCounter '1'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:730 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' , ErrorCode = '0x8027025b', processId '5144', processStatus '-532462766'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:894 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' at stage 'PostStartCheck', ErrorCode = '0x8027025b', assigned port 35088, retryCounter '0'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:948 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' with multiple retries. The last try of listening port is '35088'. See previous warnings for details.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\processmanager.cpp:167 
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\forwardinghandler.cpp:138 
[aspnetcorev2_outofprocess.dll] FORWARDING_HANDLER::~FORWARDING_HANDLER
[aspnetcorev2_outofprocess.dll] FORWARDING_HANDLER::FORWARDING_HANDLER
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:730 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' , ErrorCode = '0x8027025b', processId '10720', processStatus '-532462766'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:894 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' at stage 'PostStartCheck', ErrorCode = '0x8027025b', assigned port 45881, retryCounter '1'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:730 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' , ErrorCode = '0x8027025b', processId '12420', processStatus '-532462766'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:894 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' at stage 'PostStartCheck', ErrorCode = '0x8027025b', assigned port 31772, retryCounter '0'.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\serverprocess.cpp:948 
[aspnetcorev2_outofprocess.dll] Event Log: 'Application '/LM/W3SVC/1730955488/ROOT' with physical root 'D:\home\site\wwwroot\' failed to start process with commandline 'D:\home\site\wwwroot\Sabio.Web.Api.exe ' with multiple retries. The last try of listening port is '31772'. See previous warnings for details.' 
End Event Log Message.
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\processmanager.cpp:167 
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025b at c:\b\w\e37dd45d8cd1eaf4\modules\iisintegration\src\aspnetcoremodulev2\outofprocessrequesthandler\forwardinghandler.cpp:138 
[aspnetcorev2_outofprocess.dll] FORWARDING_HANDLER::~FORWARDING_HANDLER

I have this as a part of start up

            WebHost.CreateDefaultBuilder(args)
            .UseSetting(WebHostDefaults.DetailedErrorsKey, "true")
            .UseKestrel()
            .UseIISIntegration()

AND Locally I had implemented the bug fix for the "Current Directory" bug that I fixed with the suggest code snippet found somewhere in these issues BUT I disabled that b/c I am under the impression that is only for IN process applications.

[mhosman]

Same here!!

[fierro]

So knowing I had a test app out in Azure deployed in process, I went back to this application and change the web.config and program.css to act as if it was in process. and it works now.

I do have that fix in place to for the Current Directory Bug in place form here

I did not need that before but now i do i guess.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
            .UseSetting(WebHostDefaults.DetailedErrorsKey, "true")
            .UseIISIntegration()
            .UseIIS()
            .ConfigureAppConfiguration(ConfigConfiguration)
            .ConfigureLogging(ConfigureLogging)
            .UseStartup<Startup>();

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet"
                arguments=".\Sabio.Web.Api.dll" stdoutLogEnabled="true"
                stdoutLogFile="\\?\%home%\LogFiles\stdout" hostingModel="inprocess">
      <handlerSettings>
        <handlerSetting name="debugLevel" value="FILE,CONSOLE,TRACE,INFORMATION" />
        <handlerSetting name="debugFile" value="aspnetcore-debug.log" />
      </handlerSettings>
      <environmentVariables>
        <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Production" />
        <environmentVariable name="ASPNETCORE_HTTPS_PORT" value="44373" />
      </environmentVariables>
    </aspNetCore>
    <security></security>
  </system.webServer>
</configuration>

[AdamUCF]

I ran into this and the issue error and the issue ended up being that I was using DPAPI to store keys in the user's registry and my app pool wasn't set to load the user profile. Flipping that fixed this.

[koo9]

@fierro where do call the fix for setting the current directory?

[arkiaconsulting]

For my part, the problem had nothing to do with the host configuration, not the deployment method. It was a startup configuration method that was trying to access an Azure resource without the required permissions. I found out what the problem was by running dotnet xxxx.dll from the Kudu console. That way you get the full exception.

[ablaze8]

If someone stumbles upon this page, another possible reason is your app settings JSON file is not formatted correctly. Yes that'd do it! Open your web.config and enable logging stdoutLogEnabled="true"

[mjkent]

For someone stumbling upon this in Azure. Going to the webapp, console, then running dotnet yourdll.dll to see what is going on. In my case it was that I was using version 2.2.6, but Azure hasn't installed it yet, they are only up to 2.2.5.

[pspkshah]

I am using framework dependant deployment and @fierro solution did not worked.

My published files works locally fine. But not working in azure.

below is my web.config, I am using .net Framework 4.7 and .net core 2.2 with Abp (AspNetBoilarplate) framework.

My Program.cs file is as below.

Tried so much and worked with microsoft support as well, but issue still not resolved please suggest.