Azure-docs: Terminate Event Cannot be Re-Enabled after Disabling

[ThunderHorse]

I was able to configure my vmss to pick up terminate events via the Azure Rest API call. It appears that after you enable it if you disable by changing enable to false in the payload of the API call you cannot re-enable after doing so. So, on my scale set, I enabled terminate events, disabled, and I am unable to re-enable the scale set to get terminate events.

API Call Payload To Enable (this initially worked and subsequent calls to re-enable do not):

{ "properties":{ "virtualMachineProfile":{ "scheduledEventsProfile":{ "terminateNotificationProfile":{ "notBeforeTimeout":"PT15M", "enable":true } } } }, "location":"eastus" }

API Call Payload To Disable:

{ "properties":{ "virtualMachineProfile":{ "scheduledEventsProfile":{ "terminateNotificationProfile":{ "notBeforeTimeout":"PT15M", "enable":false } } } }, "location":"eastus" }

I know the API call is correct and working because I have been able to enable terminate events on multiple scale sets by using it.

After running the api call I make a powershell call to:

Get-AzVmss -ResourceGroupName "redacted" -VMScaleSetName "redacted"

After enabling and running that command I would expect to see:

Instead after attempting to re-enable I see this when executing the call to get vmss info:

The scheduledEventProfile never gets reset with another value after attempting to re-enable.

Docs for Enabling Terminate Notification: https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-terminate-notification

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 435c7f55-11e0-202d-cc7a-6f636f0c2a11
• Version Independent ID: 31cac987-8d27-5f7b-2cf6-c7b6b0edee48
• Content: Terminate notification for Azure virtual machine scale set instances
• Content Source: articles/virtual-machine-scale-sets/virtual-machine-scale-sets-terminate-notification.md
• Service: virtual-machine-scale-sets
• GitHub Login: @shandilvarun
• Microsoft Alias: vashan

[TravisCragg-MSFT]

@ThunderHorse Thanks for the feedback. We are actively investigating and will get back to you soon.

[mimckitt]

@shandilvarun do you know if this is a currently known limitation? I know this feature is in preview but if we are not aware of this it would be worth looking into.

[szimmerMicrosoft]

@ThunderHorse
I tried to find an example of what you are reporting.
I see some usage where scheduled events are enabled, and then disabled in the VM Scale Set API, and looking at the current state of the system I see that after enable - disable - enable the terminate notifications appear to be enabled.

I think to investigate this issue in more detail I need the resource details where you were testing this.

Can you please email me at szimmer [at] microsoft.com with the following details

1. SubId
2. resource group name
3. Vmss Name
4. Time you attempted this enable - disable -reenable
5. How you are issuing the enable / disable (powershell, cli, some client library?) and the command you used

[mimckitt]

thanks @szimmerMicrosoft !

@ThunderHorse please send the email with the requested details. You can also include me mimckitt [at] microsoft.com

I will close this and we can continue offline. We we end up needing to add any notes to the doc I will reopen and do so.

[ThunderHorse]

@szimmerMicrosoft , this should be reproducible on any scale set. I will email you details but you shouldn't need them. @MicahMcKittrick-MSFT please open this issue back up it is not yet resolved and may be useful to anyone experiencing a similar issue.

[mimckitt]

@ThunderHorse we tested this but were not able to reproduce the issue you are seeing. Hence, we can continue it offline. And this issue will still be visible on the doc regardless of if it is open or closed.

[ThunderHorse]

@MicahMcKittrick-MSFT, Ok, I sent the email over earlier today. I look forward to your response.

@szimmerMicrosoft

[szimmerMicrosoft]

@ThunderHorse I was able to repro the issue, thanks for providing the details, and thank you for reporting this issue.

I identified the bug, and will check in the fix today. We execute the update on the server side, but if that is the only change we are making we forget to persist it :^ (.

The fix should be rolled out globally within the next 30 days, if you need to work around this in the meantime if you update an unrelated property while applying the reenable scheduled events (such as changing a tag, or making any other change) then the update should be applied.

[mimckitt]

@szimmerMicrosoft would it make sense to add a note to the doc about the work around? Or leave it as is and users can reference this issue if they encounter an issue?

[szimmerMicrosoft]

I think this thread is good enough for now.
This feature does not have high usage, and this is a niche issue I think.

[ThunderHorse]

@szimmerMicrosoft , @MicahMcKittrick-MSFT, Thank you both for looking into this issue!

[ThunderHorse]

@szimmerMicrosoft , I attempted this today by changing a tag as recommended and also by adding a new tag in a second attempt. It does not appear that I can re-enable this way. Were there any other properties that may work for this? Here is an example of my payload:

{
"properties":{
"virtualMachineProfile":{
"scheduledEventsProfile":{
"terminateNotificationProfile":{
"notBeforeTimeout":"PT5M",
"enable":true
}
}
}
},
"location":"westeurope",
"Tags":{
"newTag":"blah"
}
}

[szimmerMicrosoft]

@ThunderHorse I took a look at your operation -- from the logs it looks like you need to update some other property within the virtualMachineProfile, for us to persist the scheduledEventsPolicy in the virtualMachineProfile

You could add a custom script extension which does nothing ( or any other extension)
You could update any networking settings (unfortunately I do not have a recommendation here for a harmless network setting).
You could also update your password, or customData (if you are using linux).

[ThunderHorse]

Updating a path on a powershell script extension allowed me to re-enable.

Thanks @szimmerMicrosoft

[szimmerMicrosoft]

@ThunderHorse Great.
We are still on track to complete the rollout he fix by end of next week

[szimmerMicrosoft]

@ThunderHorse Thanks again for your feedback on this feature.
Terminate events have left Preview and are now GA if that is something that would influence your adoption.