Azure-cli: Webapp:Retrieve webapp publishing profile as XML

[SeanFeldman]

Describe the bug

Cannot retrieve webapp publishing profile as XML.

To Reproduce

Running the following

az webapp deployment list-publishing-profiles `
  --name <webapp-name> `
  --resource-group <webapp-resource-group>

generates a JSON file, not XML.

Expected behavior

To be able to generate XML file, similar to what's generated in the portal when the Get publish profile button it used.

Environment summary

Cloud shell

{
  "azure-cli": "2.5.1",
  "azure-cli-command-modules-nspkg": "2.0.3",
  "azure-cli-core": "2.5.1",
  "azure-cli-nspkg": "3.0.4",
  "azure-cli-telemetry": "1.0.4",
  "extensions": {}
}

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @AzureAppServiceCLI, @antcp.

[yonzhan]

webapp

[panchagnula]

@SeanFeldman any specific reason for xml response request - CLI returns the object that the API returns which is json here. Thanks!

[SeanFeldman]

@SeanFeldman any specific reason for xml response request - CLI returns the object that the API returns which is json here. Thanks!

Indeed. When a publishing profile file downloaded from the portal, it's an XML file, not JSON. This file is used to perform the ZIP deployment of a webjob/webapp. Specifically, I use GitHub action srijken/[email protected] for that purpose. The publish profile is expected to be an XML file.

[SeanFeldman]

Also, Visual Studio based deployment relies on the this file to be an XML file.

[sitereactor]

Additionally, the official Azure App Service github actions expects the publishing profile for deploying and if you are creating the app service webapp as part of the Github Workflow its close to impossible to automate the deployment, because the az cli is not able to produce the expected xml file.

https://github.com/marketplace/actions/azure-webapp#dependencies-on-other-github-actions

[SeanFeldman]

@panchagnula any updates on this?

[panchagnula]

@SeanFeldman we will take a look at this - no ETA yet to share.

[SeanFeldman]

we will take a look at this - no ETA yet to share.

I was hoping this would be already in a validation state as it was reported a week ago...

[panchagnula]

Looking at our code CLI seems to be explicitly converting this to JSON - API returns XML since this will be a breaking change we need to handle this correctly - we might need to add a property to allow XML, so give me sometime to run this by cli folks to handle breaking changes

[SeanFeldman]

Too bad --output doesn't support XML. It would solve the problem.

[SeanFeldman]

BTW, it feels like the proper response should be XML. And if --output would be json only then the conversion would take place.

[SeanFeldman]

@panchagnula, any updates?

[panchagnula]

@SeanFeldman sorry no updates yet - I will keep this updated though - the milestone flag should be a good indicator of our tentative ETA for now. Thanks!

[panchagnula]

@yonzhan i want to understand some details here before making change to support this.
The current command returns XML to JSON since i believe CLI doesn't support XML as output - is my assumption correct is there a suggestion on how to support the request here? Thanks!

[jiasli]

Hi @panchagnula, CLI is never designed or tested with XML. Even for storage data-plane API, the XML response body is converted to JSON.

I can think of 2 solutions:

1. Make Knack compatible with XML - serialize Python object -> XML, but this will of course take some development efforts.
2. Wrap the response in JSON when --xml parameter is specified in az webapp deployment list-publishing-profiles, so that we don't introduce breaking change. For example:
   { "value": "XML Content Here" }

+ PM @achandmsft @chenlomis

[jiasli]

Checked the source code of az webapp deployment list-publishing-profiles, the response body from the underlying SDK is already XML. It is actually appservice command module that is doing the XML -> Python object conversion.

https://github.com/Azure/azure-cli/blob/3cd3b456dc5f18bca38700431cf3b8ca477d820d/src/azure-cli/azure/cli/command_modules/appservice/custom.py#L1874

To return the raw XML, method list_publish_profiles can simply return full_xml and tell the user to append --output tsv:

> az webapp deployment list-publishing-profiles --ids /subscriptions/0b1f6471-1bf0-4dda-aec3-cb9272f09590/resourceGroups/-test/providers/Microsoft.Web/sites/test --output tsv
<publishData><publishProfile profileName="test - Web Deploy" ...

Without --output tsv, the XML will be wrapped in a JSON string:

> az webapp deployment list-publishing-profiles --ids /subscriptions/0b1f6471-1bf0-4dda-aec3-cb9272f09590/resourceGroups/-test/providers/Microsoft.Web/sites/test
"<publishData><publishProfile profileName=\"test - Web Deploy\" ...

[panchagnula]

Checked the source code of az webapp deployment list-publishing-profiles, the response body from the underlying SDK is already XML. It is actually appservice command module that is doing the XML -> Python object conversion.

https://github.com/Azure/azure-cli/blob/3cd3b456dc5f18bca38700431cf3b8ca477d820d/src/azure-cli/azure/cli/command_modules/appservice/custom.py#L1874

To return the raw XML, method list_publish_profiles can simply return full_xml and tell the user to append --output tsv:

> az webapp deployment list-publishing-profiles --ids /subscriptions/0b1f6471-1bf0-4dda-aec3-cb9272f09590/resourceGroups/test/providers/Microsoft.Web/sites/test --output tsv
<publishData><publishProfile profileName="..." ...>...</publishProfile>...</publishData>

@jiasli that is correct - this command was infact authored by the core-cli team this way a long time back & assume it was done this way was because the command if run without the xml to JSON conversion minus the --output tsv would lead to errors - hence my question here is is there a way to make the output tsv the default output type for just this command instead of asking the customers to add that at the end of the command. Thanks!

[jiasli]

Is there a way to make the output tsv the default output type for just this command instead of asking the customers to add that at the end of the command.

Fortunately, yes! But it is a little bit hacky. I have drafted a PR in my own repo https://github.com/jiasli/azure-cli/pull/8 and you may follow the same pattern to submit an official PR and fix related test cases.

Perhaps we can avoid introducing breaking change by specifying --xml. 😉

[SeanFeldman]

Perhaps we can avoid introducing breaking change by specifying --xml.

That'd be ideal.

[SeanFeldman]

@panchagnula looking at the documentation, it doesn't reflect the added --xml option.
Is there a documentation PR or how to know that the fix has been deployed? I've looked at the S173 milestone but it's still opened. Thank you.

[justinyoo]

I'm using azure CLI v2.11.1. I added the --xml option. However, it returns escaped string. Do we need to handle those escaped characters by ourselves?

[frank-fischer-by]

It's "slightly" confusing.
You have to combine --xml with --output tsv to get a useable format (at least that's what we do ....)

[justinyoo]

That's far better than what I do:

az functionapp deployment list-publishing-profiles -g <resource_group> -n <functionapp_name> --xml | sed -e 's|\"<|<|g' | sed -e 's|>\"|>|g' | sed -e 's|\\"|"|g' | sed -e 's|\\\\|\\|g'

[panchagnula]

It's "slightly" confusing.
You have to combine --xml with --output tsv to get a useable format (at least that's what we do ....)

@Kotasudhakarreddy is this true? I thought your change took care of the hack to not needing the --output tsv. Can you take a look or update the help document as well accordingly? Thanks!

[justinyoo]

@panchagnula This is what I ran:

az webapp deployment list-publishing-profiles -g <resource_group_name> -n <webapp_name> --xml 

This is what I get:

"<publishData><publishProfile profileName=\"<webapp_name> - Web Deploy\" publishMethod=\"MSDeploy\" publishUrl=\"<webapp_name>.scm.azurewebsites.net:443\" msdeploySite=\"<webapp_name>\" userName=\"$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile><publishProfile profileName=\"<webapp_name> - FTP\" publishMethod=\"FTP\" publishUrl=\"ftp://waws-prod-se1-001.ftp.azurewebsites.windows.net/site/wwwroot\" ftpPassiveMode=\"True\" userName=\"<webapp_name>\\$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile><publishProfile profileName=\"<webapp_name> - ReadOnly - FTP\" publishMethod=\"FTP\" publishUrl=\"ftp://waws-prod-se1-001dr.ftp.azurewebsites.windows.net/site/wwwroot\" ftpPassiveMode=\"True\" userName=\"<webapp_name>\\$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile></publishData>"

As you can see, I've got all double quotes escaped, and leading and trailing double quotes. I suspect that the original XML string is JSON serialised with that ---xml option.

[panchagnula]

@panchagnula This is what I ran:

az webapp deployment list-publishing-profiles -g <resource_group_name> -n <webapp_name> --xml 

This is what I get:

"<publishData><publishProfile profileName=\"<webapp_name> - Web Deploy\" publishMethod=\"MSDeploy\" publishUrl=\"<webapp_name>.scm.azurewebsites.net:443\" msdeploySite=\"<webapp_name>\" userName=\"$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile><publishProfile profileName=\"<webapp_name> - FTP\" publishMethod=\"FTP\" publishUrl=\"ftp://waws-prod-se1-001.ftp.azurewebsites.windows.net/site/wwwroot\" ftpPassiveMode=\"True\" userName=\"<webapp_name>\\$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile><publishProfile profileName=\"<webapp_name> - ReadOnly - FTP\" publishMethod=\"FTP\" publishUrl=\"ftp://waws-prod-se1-001dr.ftp.azurewebsites.windows.net/site/wwwroot\" ftpPassiveMode=\"True\" userName=\"<webapp_name>\\$<webapp_name>\" userPWD=\"<password>\" destinationAppUrl=\"https://<webapp_name>.azurewebsites.net\" SQLServerDBConnectionString=\"\" mySQLDBConnectionString=\"\" hostingProviderForumLink=\"\" controlPanelLink=\"http://windows.azure.com\" webSystem=\"WebSites\"><databases /></publishProfile></publishData>"

As you can see, I've got all double quotes escaped, and leading and trailing double quotes. I suspect that the original XML string is JSON serialised with that ---xml option.

Thanks @justinyoo to confirm if you run the same command with --output tsv at the end the return object is formatted as true XML?

[justinyoo]

@panchagnula If I put the --output tsv at the end, then the escaping goes away. I mean, it returns the proper XML string.