Azure-docs: VM creation from Shared Image Gallery does not pass Purchase Plan details

[jamesbannan]

I have encountered a bug in Azure DevTest Labs when creating a new VM which is based on a Shared Image Gallery image which includes Purchase Plan information.

I have created a custom image using the CIS Red Hat Enterprise Linux 7 image from the Azure Marketplace. This image has been stored as an image within a Shared Image Gallery (which is also attached to a DevTest Labs instance), and the shared image includes all the purchase plan information (publisher, offer, SKU).

I am seeing the following behaviour when using this image:

1. I can successfully create a new VM from this image (outside DevTest Labs)
2. I can successfully deploy a new VM within DevTest Labs from the original marketplace image
3. I cannot deploy a new VM within DevTest Labs using the shared image

When trying to deploy a new VM within DevTest Labs using the shared image, the deployment constantly fails with the following error:

{
    "status": "Failed",
    "error": {
        "code": "ResourceDeploymentFailure",
        "message": "The resource operation completed with terminal provisioning state 'Failed'.",
        "details": [
            {
                "code": "VMMarketplaceInvalidInput",
                "message": "Creating a virtual machine from Marketplace image or a custom image sourced from a Marketplace image requires Plan information in the request. VM: '<VM_ID>."
            }
        ]
    }
}

The purchase plan information is therefore not being passed from the shared image to the DevTest VM deployment. However, clearly it is possible to deploy a VM in DevTest Labs from a Marketplace image which includes purchase plan information, because the original CIS RHEL image works fine.

All of the recent examples for deploying to DevTest Labs are using an undocumented API version (2018-10-15-preview) which is making troubleshooting and experimentation very difficult.

Essentially at this point, it does not appear to be possible to deploy a new VM in Azure DevTest Labs from a Shared Image Gallery image which contains purchase plan information.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 965a0db2-4c54-554a-b972-3d0473eb9a0e
• Version Independent ID: 1ce1182f-c558-b1e8-cb51-35155e8cea76
• Content: Add a VM using a shared image in Azure DevTest Labs - Azure DevTest Labs
• Content Source: articles/devtest-labs/add-vm-use-shared-image.md
• Service: devtest-lab
• GitHub Login: @spelluru
• Microsoft Alias: spelluru

[jamesbannan]

Further to this, looking at the auto-generated ARM template when deploying the CIS RHEL Marketplace Image to DevTest Labs, the properties section includes the following information, including the planId:

"properties": {
        "labVirtualNetworkId": "[variables('labVirtualNetworkId')]",
        "notes": "CIS Red Hat Enterprise Linux 7 Benchmark L1",
        "galleryImageReference": {
          "offer": "cis-rhel-7-v2-2-0-l1",
          "publisher": "center-for-internet-security-inc",
          "sku": "cis-rhel7-l1",
          "osType": "Linux",
          "version": "latest"
        },
        "planId": "cis-rhel7-l1",

Looking at the same deployment template but specifying the shared image as the source, the properties section includes the sharedImageId property but does not include the planId, nor is there any option in the deployment to include it.

If I use an ARM template deployment for the shared image and include the planId property, the deployment simply fails with the same error message:

{
    "status": "Failed",
    "error": {
        "code": "ResourceDeploymentFailure",
        "message": "The resource operation completed with terminal provisioning state 'Failed'.",
        "details": [
            {
                "code": "VMMarketplaceInvalidInput",
                "message": "Creating a virtual machine from Marketplace image or a custom image sourced from a Marketplace image requires Plan information in the request. VM: '<VM_ID>"
            }
        ]
    }
}

[BhargaviAnnadevara-MSFT]

@jamesbannan Thanks for the question and the details! We're investigating this and will get back to you shortly.

[BhargaviAnnadevara-MSFT]

@jamesbannan Thank you for bringing this to our attention. Our product group is investigating the issue further.

I will get back to you once I have an update. Thanks for your patience.

[leovms]

@jamesbannan, did you try to create a VM from the custom image itself? I can repro your case with a custom image. So, I am not sure if it is isolated to SIG images. Can you please confirm? Also, can you tell me if you deprovisioned the VM when creating the custom image?

[jamesbannan]

@leovms Yes, I was able to create a VM outside DevTest Labs from the custom image without any problem. The image was created using Hashicorp Packer, and the final step deprovisions the VM:

{
    "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
    "inline": [
      "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
    ],
    "inline_shebang": "/bin/sh -x",
    "type": "shell"
}

Within DevTest Labs, the custom image can only be applied using SIG because it's a managed image, not a VHD in a storage account. I'd need to re-build the image to test that scenario. However the plan is definitely to be using managed images, not VHD-based images.

[leovms]

@jamesbannan when I refer to custom image, I mean a DTL custom image. So, when you create the original VM from the gallery image, you mentioned that you created a custom image and I'm assuming this was a DTL custom image. If so, when you then create a new VM from that custom image, within DTL, does that work? I find it does not and I'm looking for confirmation from you that it also does not work, as it confirms I'm following the correct repro steps.

Similarly, when you configure the SIG image, do you specify the Purchase Plan information? Even from SIG, I am able to repro your issue if I don't configure the Purchase Plan in the SIG, when I try to create a VM from that SIG image.

Having said all that, I am confirming that DTL does not pass along the Purchase Plan information to the underlying call when creating the VM. I am fixing that now. It will take a few days for the fix to roll out. In the meantime, I would appreciate that you confirm my observations.

[jamesbannan]

@leovms I see - my mistake. No, I didn't create the custom image within DTL and I haven't tried. The process I have for building the image is using Azure DevOps and Packer, and the azure-arm builder wasn't able to provision a new VM within DTL and capture it as a custom image. So my approach has been to use Packer to provision a VM outside DTL, capture it to a managed image, and then bring into SIG as a new version. The SIG is attached to DTL.

There is a recent PR in the Packer project to add integration with DTL, but I've not checked out the functionality included in the merge.

To answer your other questions, Yes when I configure the SIG image I definitely include the purchase plan information. When DTL first failed to apply the custom image via SIG I hadn't included the purchase plan information, and I thought that this was why it failed. I went back and added the purchase plan information, and that was when I discovered that the information wasn't being passed through to DTL, regardless of the information supplied in the SIG image.

[leovms]

@jamesbannan, thank you for reporting this and the details on the issue. The fix is currently being validated and should be rolling out sometime after that. It takes about 1 week to roll out to all regions.

[BhargaviAnnadevara-MSFT]

@jamesbannan The fix should have been rolled out to all regions by now. Please retry and let us know if you see any issues. Thanks for your patience!

[jamesbannan]

Hi @BhargaviAnnadevara-MSFT - thank you for the update. I have tested a deployment of a custom RHEL image from a shared image gallery attached to DevTest Labs and it worked perfectly :-) I'll close this call now - thanks all for your assistance with this issue.

[BhargaviAnnadevara-MSFT]

@jamesbannan Thanks for confirming! We're glad we could resolve your issue!