Azure-sdk-for-python: Creating Disk: Unable to serialize value <DiskCreateOption object>

[MingweiSamuel]

I can't figure out how to create a disk. It seems that it isn't handling the DiskCreateOption correctly?

https://docs.microsoft.com/en-us/python/api/azure.mgmt.compute.v2017_03_30.operations.disksoperations?view=azure-python#create-or-update

Code:

#!/usr/bin/env python
import argparse

import os
import subprocess

from azure.common.client_factory import get_client_from_auth_file

from azure.mgmt.compute import ComputeManagementClient
from azure.mgmt.compute.models import DiskCreateOption
# import azure_utils

AUTH_PATH = os.getenv('HOME') + '/.azure/azure_credentials.json'
compute_client = get_client_from_auth_file(ComputeManagementClient, auth_path=AUTH_PATH)

print(DiskCreateOption.empty)

disk_config = {
    'location': 'westus',
    'sku': {
        'name': 'Premium_LRS'
    },
    'os_type': 'linux',
    'creation_data': {
        'create_option': 'Empty'
    },
    'disk_size_gb': 100
}
disk_creation = compute_client.disks.create_or_update(
    'asdf',
    'testdisk',
    disk_config
)
disk = disk_creation.result()

Output:

Traceback (most recent call last):
  File "lib/create_disk.py", line 32, in <module>
    disk_config
  File "/home/msamuel/.local/lib/python2.7/site-packages/azure/mgmt/compute/v2017_03_30/operations/disks_operations.py", line 89, in create_or_update
    body_content = self._serialize.body(disk, 'Disk')
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 476, in body
    return self._serialize(data, data_type, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 399, in _serialize
    target_obj, data_type, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 611, in serialize_data
    return self._serialize(data, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 422, in _serialize
    orig_attr, attr_type, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 611, in serialize_data
    return self._serialize(data, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 422, in _serialize
    orig_attr, attr_type, **kwargs)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/serialization.py", line 609, in serialize_data
    SerializationError, msg.format(data, data_type), err)
  File "/home/msamuel/.local/lib/python2.7/site-packages/msrest/exceptions.py", line 48, in raise_with_traceback
    raise error
msrest.exceptions.SerializationError: Unable to serialize value: <azure.mgmt.compute.v2017_03_30.models.compute_management_client_enums.DiskCreateOption object at 0x7f695897ec10> as type: 'DiskCreateOption'., EnumBadKeyError: Enumeration keys must be strings: <azure.mgmt.compute.v2017_03_30.models.compute_management_client_enums.DiskCreateOption object at 0x7f695897ec10>

[MingweiSamuel]

Removing the sku and os_type to match https://github.com/Azure/azure-sdk-for-python/blob/77485080010ced93c010f1d4f557b90bb7b3cc02/azure-mgmt-compute/tests/test_mgmt_managed_disks.py#L39-L56 has the same issue

[lmazuel]

That's weird, I try to look at that asap. In the meantime, could you confirm your package version number of azure-mgmt-compute and msrest?
Thank you

[MingweiSamuel]

I have

azure==2.0.0  # (what I pip installed)
azure-mgmt==2.0.0
msrest==0.4.29
msrestazure==0.4.31

[lmazuel]

azure 2.0.0 installs azure-mgmt 1.0.0, so I guess at some point you either just did "pip install azure" or updated azure-mgmt?
Could you give me the version of azure-mgmt-compute, that the one that matters.
Thanks

[MingweiSamuel]

Oops, its azure-mgmt-compute==1.0.0

[lmazuel]

So, API Version 2017-03-30 did not exists in 1.0.0, so something's wrong here :/. It was introduced in 2.0.0. Also, latest stable is 3.0.1, but you should use 4.0.0rc2 which is close to stable and can be relied on.
Could you test more recent packages and/or figure out the inconsistency here?
Thanks,

[MingweiSamuel]

Ok, I will let you know tomorrow

[MingweiSamuel]

Uninstalled everything and installed the latest of the packages I need:

azure-common==1.1.11
azure-mgmt-compute==...
azure-mgmt-network==2.0.0rc2 # edit -> 2.0.0rc1
azure-mgmt-nspkg==2.0.0
azure-mgmt-resource==2.0.0rc1 # edit -> 1.2.2
azure-nspkg==2.0.0
msrest==0.4.29
msrestazure==0.4.31

Then for mgmt-compute, azure-mgmt-compute==3.1.0rc3 or 3.0.1 or 2.1.0 all had the original issue.

Using azure-mgmt-compute==4.0.0rc2 had a different exception, something relating to profiles:

Traceback (most recent call last):
  File "lib/test_swap_osdisk.py", line 36, in <module>
    disk_creation = compute_client.disks.create_or_update(
  File "/home/msamuel/.local/lib/python2.7/site-packages/azure/mgmt/compute/compute_management_client.py", line 181, in disks    api_version = self._get_api_version('disks')
  File "/home/msamuel/.local/lib/python2.7/site-packages/azure/profiles/multiapiclient.py", line 70, in _get_api_version
    current_profile = KnownProfiles.default.value.definition()
AttributeError: 'DefaultProfile' object has no attribute 'value'

Using azure-mgmt-compute==4.0.0rc1 works as expected, creates the disk, no errors or warnings. So I will try that for now. Let me know if you need any other info. And thanks for helping with this

Edit:
Downgraded azure-mgmt-resource==1.2.2 and azure-mgmt-network==2.0.0rc1 b/c was getting the same error as azure-mgmt-compute==4.0.0rc2 when trying to make a resource group or vnets

[lmazuel]

Thanks @MingweiSamuel that's very valuable! I try to look into the details now asap

[MingweiSamuel]

I suspect this might be related to https://pypi.org/project/enum/ being installed on the machine I was using, will test tomorrow

[lmazuel]

Hi @MingweiSamuel
Yes, I confirm this is incompatible with the enum package. For backward compat, our runtime uses enum34 which is a real backport of the std lib of Py3.
This solves the problem you have with 4.0.0rc2

[lmazuel]

Tested with 3.0.1 with this simple serialization test:

from azure.mgmt.compute.v2017_03_30.models import *
disk = Disk.from_dict({
   'location': 'eastus',
   'disk_size_gb': 20,
   'creation_data': {
      'create_option': DiskCreateOption.empty
   }
})
disk.creation_data.create_option # will show  <DiskCreateOption.empty: 'Empty'>

And works.

So my current guess is that all of this is created by the unexpected enum package (all failures are related to enum). I'm closing the issue, please still comment on it if you're able to reproduce without enum installed.
Thanks!