Azure-cli: VM diagnostics

[prafulbusa]

By following the article below, i am trying to enable the diagnostics settings using Azure CLI instead of manually for my Azure VM. however the script runs fine but it is not enabling the diagnostics setting and also no error message. please help me out with the solution. https://docs.microsoft.com/en-us/cli/azure/vm/diagnostics?view=azure-cli-latest&viewFallbackFrom=azure-cli-latest.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior.

Expected behavior
A clear and concise description of what you expected to happen.

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)

Additional context
Add any other context about the problem here.

[adewaleo]

Hello @prafulbusa, what does the cli return when you run the command and what did you expect. As much information as possible / reasonable would be helpful. Thanks!

[prafulbusa]

Unable to set guest-level monitoring disagnostic from azure cli.
Cx is automating the creation of VMs using azure cli and one of the things that he was trying to do is set the diagnostics as shown below. When he look at the result from the Azure portal, it shows the diagnostic being set but the the save button is enabled meaning the diagnostic setting has not been saved and has to be done manually. I am not clear as to why this is occurring and what should be done to not end up in such a state.

[adewaleo]

@prafulbusa as we have discussed:

• The fact that the save button is not greyed out and is clickable does not mean the diagnostics extension has not been set. If you create a linux vm and check the diagnostics tab on portal or run az vm show before running az vm diagnostics set, you will see that there is a difference before and after. Even if you click the save button, it can be clicked again after the same extension settings have been applied.
• A user can verify that diagnostics have been properly set by looking at the storage account specified in their settings. They should see that diagnostics data is being set to it.
• I will update the documentation example for az vm diagnostics set. Some shell variables were not properly referenced.
• UPDATED: You can use az vm diagnostics set or az vm extensions set to set diagnostics for a windows vm in the same way. az diagnostics set can be thought of as a wrapper around az vm extension set. Simply supply the correct settings and protected settings. You can run az vm diagnostics get-default-config --is-windows-os to get the default config info for a windows vm. This article has more information about setting diagnostics on a windows vm.

[tjprescott]

This syntax {foo} was not used to imply it was a shell variable. It is the standard placeholder syntax we use in examples. However, since the entire example presents itself as a script, it would make sense to use bash variable syntax.

[jagan-at-vi]

I have the same issue - az cli setting of diagnostics does not work for windows unless the save is performed in the portal for that vm's diagnostic setting. (BTW, not an issue with Linux VMs.).

my_vm_resource_id=$(az vm show -g $my_resource_group -n $my_win_vm --query "id" -o tsv)
protected_settings="{'storageAccountName': '${my_diagnostic_storage_account}', \
'storageAccountSasToken': '${storage_sastoken}'}"

default_config=$(az vm diagnostics get-default-config --is-windows-os \
| sed "s#__DIAGNOSTIC_STORAGE_ACCOUNT__#$my_diagnostic_storage_account#g" \
| sed "s#__VM_OR_VMSS_RESOURCE_ID__#$my_vm_resource_id#g")

storage_sastoken=$(az storage account generate-sas \
--account-name $my_diagnostic_storage_account --expiry 2037-12-31T23:59:00Z \
--permissions wlacu --resource-types co --services bt -o tsv)

az vm extension set --publisher Microsoft.Azure.Diagnostics \
--name IaaSDiagnostics \
--protected-settings "${protected_settings}" \
--settings "${default_config}" \
--version "1.5" \
--resource-group $my_resource_group --vm-name $my_win_vm

[jagan-at-vi]

Outputs using
az vm show -n gr-sim-win -g gr-rg-westcentralus --query "resources[]"

[adewaleo]

@jagan-at-vi, I will look into this. You can remove / update the txt files. They have your subscriptions ids in them.

@tjprescott, ah I see, this is good to know. I wasn't aware of this format

[adewaleo]

@jagan-at-vi, I could reproduce this issue with windows VMs. I, too, couldn't set the extension. Even after fixing / altering the default config settings. It seems, this might be a service side issue.

I reached out to the extension's documentation authors for help.

[adewaleo]

The issue I see is that the agent does not seem to create any of the tables in the specified storage account that are needed for the APIs to retrieve rows relating to counter values.

@jagan-at-vi. Yes, I understand the extension is not getting successfully provisioned. When I asked what error you got, I meant what the output / result of executing the diagnostics set / extensions set command was? I wanted to confirm that it is a similar error to what I was getting. I am sure if you run az vm show -n yourVMName -g yourRG you will see that the extension's provisioning state is "failed".

[jagan-at-vi]

See attached for output of az vm show

---

From: adewaleo [e-mail]
Sent: Thursday, January 3, 2019 1:05:24 PM
To: Azure/azure-cli
Cc: Jagan Jagannathan; Mention
Subject: Re: [Azure/azure-cli] VM diagnostics (#8108)

The issue I see is that the agent does not seem to create any of the tables in the specified storage account that are needed for the APIs to retrieve rows relating to counter values.
…
________________________________ From: adewaleo [email protected][e-mail] Sent: Thursday, January 3, 2019 10:06:46 AM To: Azure/azure-cli Cc: Jagan Jagannathan; Mention Subject: Re: [Azure/azure-cli] DNM [vm] update diagnostics set example. Fix bug in diagnostics get-default-config --is-windows-os (#8117https://github.com/Azure/azure-cli/pull/8117) Tried the latest default_config per above and not having success in seeing even tables being created in the storage account @jagan-at-vihttps://github.com/jagan-at-vihttps://github.com/jagan-at-vi, what error did you get in the CLI. There seems to be an issue with setting the extension using the CLI. This is why I placed a do not merge label on the PR. I will at least try to find some work around and mention it in 8108. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#8117 (comment)https://github.com/Azure/azure-cli/pull/8117#issuecomment-451227328>, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Amli8dxmsWusSdcy3zhFBI77HyDmSctTks5u_ka2gaJpZM4ZfNRA.

________________________________ The contents of this message, together with any attachments, are intended only for the use of the individual or entity to which they are addressed and may contain confidential information. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please contact the sender immediately and permanently delete or destroy the material/information.

@jagan-at-vihttps://github.com/jagan-at-vi. Yes, I understand the extension is not getting successfully provisioned. When I asked what error you got, I meant what the output / result of executing the diagnostics set / extensions set command was? I wanted to confirm that it is a similar error to what I was getting. I am sure if you run az vm show -n yourVMName -g yourRG you will see that the extension's provisioning state is "failed".

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-cli/issues/8108#issuecomment-451276824, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Amli8U1ZgjIsWAz6U2JtjgK9OAP6qgDKks5u_nCUgaJpZM4ZbXa8.

--

---

The contents of this message, together with any attachments, are intended only for the use of the individual or entity to which they are addressed and may contain confidential information. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please contact the sender immediately and permanently delete or destroy the material/information.

[adewaleo]

@jagan-at-vi I don't see any attached output of vm show. What might be more helpful in this case is the error you get when you run the extension set / diagnostics set command.

Also, this issue appears to be a service side issue and a CLI issue. Hence, the updated labels. I have reached out to a member of the Windows Azure diagnostics team / owners.

[jagan-at-vi]

az_vm_show.txt

[adewaleo]

The file you sent me shows that the extension was provisioned successfully. If this is truly the case, please make sure you are inspecting the correct storage account and that you are doing so properly.

As I have mentioned previously, there is an issue where attempting to set the diagnostics extension on a windows vm fails. I am waiting for feedback / help from @brahmnes who works on the extension, as this is beyond a CLI problem.

[jagan-at-vi]

Ok - will wait for further word

(Looking at the correct storage account)

.........................................
Sent from mobile device

---

From: adewaleo [e-mail]
Sent: Thursday, January 3, 2019 14:31
To: Azure/azure-cli
Cc: Jagan Jagannathan; Mention
Subject: Re: [Azure/azure-cli] VM diagnostics (#8108)

The file you sent me shows that the extension was provisioned successfully. If this is truly the case, please make sure you are inspecting the correct storage account and that you are doing so properly.

As I have mentioned previously, there is an issue where attempting to set the diagnostics extension on a windows vm fails. I am waiting for feedback / help from @brahmneshttps://github.com/brahmnes who works on the extension, as this is beyond a CLI problem.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-cli/issues/8108#issuecomment-451298515, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Amli8exvjm38ar6QRhyGDbpLOr2c_ABNks5u_oTFgaJpZM4ZbXa8.

--

---

The contents of this message, together with any attachments, are intended only for the use of the individual or entity to which they are addressed and may contain confidential information. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please contact the sender immediately and permanently delete or destroy the material/information.

[adewaleo]

@jagan-at-vi, FYI, The work around that worked for me was to go on the portal and click save after the CLI diagnostics / extension command failed to successfully provision the extension. After clicking save, the extension was provisioned successfully and the storage account started receiving diagnostics data. I used a configuration identical to the config found in PR #8117 for the windows diagnostics extension.

[jagan-at-vi]

Yes that works for me as well except I am trying to avoid any user intervention. Thanks for your help

.........................................
Sent from mobile device

---

From: adewaleo [e-mail]
Sent: Thursday, January 3, 2019 14:44
To: Azure/azure-cli
Cc: Jagan Jagannathan; Mention
Subject: Re: [Azure/azure-cli] VM diagnostics (#8108)

@jagan-at-vihttps://github.com/jagan-at-vi, FYI, The work around that worked for me was to go on the portal and click save after the CLI diagnostics / extension command failed to successfully provision the extension. After clicking save, the extension was provisioned successfully and the storage account started receiving diagnostics data. I used a configuration identical to the config found in PR #8117https://github.com/Azure/azure-cli/pull/8117 for the windows diagnostics extension.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-cli/issues/8108#issuecomment-451301499, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Amli8Xc-X2UrQL2bPk4OEkka0G_NWv14ks5u_oewgaJpZM4ZbXa8.

--

---

The contents of this message, together with any attachments, are intended only for the use of the individual or entity to which they are addressed and may contain confidential information. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please contact the sender immediately and permanently delete or destroy the material/information.

[adewaleo]

@prafulbusa and @jagan-at-vi
The newly added windows diagnostics example and updated diagnostics config in pull request #8117 work work correctly for setting diagnostics on a vm. The example also shows how one can use a storage account key as an alternative to using a storage account sas token.

The issue is that the WAD extension has strict expectations of the sas token's format. Moreover, it gives poor errors if setting the diagnostics extension fails. The status logs on the VM were helpful in coming up with a solution (thanks @brahmnes).

For the vm diagnostics set command to work properly, the sas token should:

• be enabled for the 'container (c)' and 'object (o)' resource type
• be enabled for 'blob (b)' and 'table (t)' services. This means your storage account must support blobs and tables
• be generated with 'acuw' permissions. i.e. 'add', 'create', 'update' and 'write'
• permit only HTTPS

I did not try slight variations of this. Before using these settings I got cryptic warnings such as:
"The given key was not present in the dictionary." (likely due to a bug in the parsing logic of the sastoken) and "No primary storage account" (likely when the extension rejects the token for not having the desired attributes). @brahmnes, work should be done to give richer / clearer errors when sastoken parsing fails.

When the pr #8117 is merged, I will close this issue.

Please let me know if you have any more questions @jagan-at-vi and @prafulbusa. Thanks so much for your patience!

cc: @yugangw-msft

[kingweo]

I just met alike issue when I enabled diagnostics agent with the cli command.
I let 2 windows vm enable diagnostics use the same storage account, they're enable succeed. but 2 vm display the same metric graph from metric of first VM.

[adewaleo]

@qwordy please can you follow up with this when you have time?

@kingweo I am no longer on the CLI team.
What do you mean by the 2 vms display the same metric graph? My tip for now is to verify the settings for the two VMs are different even though they are targeting the same storage account. Did you try enabling diagnostics with the portal and compare the settings via portal with the settings you used in the CLI?

Also if the CLI is passing your diagnostics settings correctly, then it is not a CLI issue.

[qwordy]

I will follow up with this issue.

[kingweo]

@qwordy please can you follow up with this when you have time?

@kingweo I am no longer on the CLI team.
What do you mean by the 2 vms display the same metric graph? My tip for now is to verify the settings for the two VMs are different even though they are targeting the same storage account. Did you try enabling diagnostics with the portal and compare the settings via portal with the settings you used in the CLI?

Also if the CLI is passing your diagnostics settings correctly, then it is not a CLI issue.

Thank you for quick reply.
I just found the reason of this issue. I used old version of az cli to set diagnostics.
As this example from az cli refrence as below:
default_config=$(az vm diagnostics get-default-config | sed "s#__DIAGNOSTIC_STORAGE_ACCOUNT__#$my_diagnostic_storage_account#g" | sed "s#__VM_OR_VMSS_RESOURCE_ID__#$my_vm_resource_id#g")
You should be know it was not be found that the string "__VM_OR_VMSS_RESOURCE_ID__" in parts of windows os from _vm_diagnostics_templates.py with the old version.
I hope someone meet the same issue and he can upgrade his az cli to latest version for resolve it correctly.