Salt: Azure managed storage

[sumeetisp]

Description of Issue/Question

How to provision azure vm's with managed disks/storage?

[Ch3LL]

Are you using azure or azure arm salt-cloud driver?

[sumeetisp]

Yes

Thanks,
Sumeet Salvankar

On 08-Dec-2017 20:58, "Megan Wilhite" [e-mail] wrote:

Are you using azure or azure arm salt-cloud driver?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-350291023,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHg-EyjRi6ZUYSm1Nj5IRQk_NILPlU0-ks5s-VWbgaJpZM4Q7FX5
.

[sumeetisp]

Sorry... I am using azure arm cloud driver

Thanks,
Sumeet Salvankar

On 08-Dec-2017 23:47, "sumeet salvankar" [e-mail] wrote:

Yes

Thanks,
Sumeet Salvankar

On 08-Dec-2017 20:58, "Megan Wilhite" [e-mail] wrote:

Are you using azure or azure arm salt-cloud driver?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-350291023,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHg-EyjRi6ZUYSm1Nj5IRQk_NILPlU0-ks5s-VWbgaJpZM4Q7FX5
.

[Ch3LL]

Thanks for the information. parsing the azure arm docs i'm only seeing the ability to clean up the disks?

ping @saltstack/team-cloud do you know of a way to manage the disks with azure arm specifically?

[gtmanfred]

There is not a way to directly manage the volumes, only in reference to building the vm and volume together.

[sumeetisp]

I am not talking abt managing volumes.

My question was about managed and unmanaged volumes provided by azure.

When we specify storage account in azure salt profile the VM that is
provisioned gets os volume from unmanaged storage.

And when we provision from azure console without specifying any storage
account gets azure managed disk.

I tried excluding storage account from profile but storage account is a
mandatory parameter.

So my question is how can I provision VM with managed disk's.
Thanks,
Sumeet Salvankar

On 13-Dec-2017 21:41, "Daniel Wallace" [e-mail] wrote:

There is not a way to directly manage the volumes, only in reference to
building the vm and volume together.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-351438829,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHg-Ez82ZlWEzcy-ALnGl4oPQiH7TkjGks5s__cqgaJpZM4Q7FX5
.

[gtmanfred]

@techhat can you comment on this since you are more familiar with the azurearm driver?

Thanks,
Daniel

[SJern]

Hi all,

The Azure documentation says, "We recommend that you use Azure Managed Disks for new VMs, and that you convert your previous unmanaged disks to managed disks, to take advantage of the many features available in Managed Disks."
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/about-disks-and-vhds

It would be great if we can figure out how to provision managed disks in salt. Thanks!

[techhat]

Looks like in volumes you can set create_option to image, attach, or empty. But not all of the required options for managed disks are available in the drive right now. For future reference: https://docs.microsoft.com/en-us/python/azure/python-sdk-azure-samples-managed-disks?view=azure-python

[sumeetisp]

According to docs point to by @techhat ,
"A Managed Disk is created implicitly when creating VM from an OS image in Azure. In the storage_profile parameter, os_disk is now optional and you don't have to create a storage account as required precondition to create a Virtual Machine."

But storage account is currently mandatory in salt azure profile. Also the default value of os_disk is set to 30.

Also the os disk itself should be provisioned as managed disk not only the volumes that are mentioned in the volumes section of salt azure profile.

[SJern]

@sumeetisp Do you know how to override the default os_disk size in the vm profile?

[sumeetisp]

There are certain parameters which are not mentioned in documentation.
You can have a look here

You need to mention os_disk_size_gb in your azure profile and its value in gb

[SJern]

Thanks, @sumeetisp! Awesome, worked like a charm!

I also noticed that any volume that is mentioned in the volumes section of the salt profile is not mounted. I always have to ssh into the machine and manually mount each of them myself. Any of you having the same issue?

[nicholasmhughes]

@sumeetisp @SJern PR #45187 changes to default to managed disks, with the ability to use unmanaged disks if desired. Note that it's not entirely clear in the MS docs, but you can't mix managed and unmanaged disks on an instance.

[SJern]

@nicholasmhughes Thanks!

[sumeetisp]

I see some function related to managed disk's in develop branch. Have you added managed disk's support?

[sumeetisp]

@SJern volumes will not get mounted automatically. They will only get attached. U may have to use mount module or write script for mounting them. Can be a part of your orchestration.

[sumeetisp]

@nicholasmhughes Please highlight profile values that need to be set for provisioning vm with managed disk.I can see "vhd" with value "unmanaged" can be specified in profile. Any other changes/aditions?

[nicholasmhughes]

@sumeetisp managed disks are the new default in the module, following guidance from Microsoft. You would explicitly set vhd to unmanaged as you noted in order to use blob storage.

[sumeetisp]

The output returned is missing some values like private and public ip.

        ----------
        hardware_profile:
            ----------
            vm_size:
                Basic_A1
        id:
            8753a40f-c73d-44c2-b8d4-c7f938484d4b
        image:
            RedHat|RHEL|7.2|latest
        location:
            eastus
        name:
            testazurevol
        network_profile:
            ----------
            network_interfaces:
                |_
                  ----------
                  id:
                      /subscriptions/1a98da19-9471-4a6a-b5f6-bed6683123a2/resourceGroups/AzureARM-SanRamon-New-ResGrp/providers/Microsoft.Network/networkInterfaces/testazurevol-iface0
        os_profile:
            ----------
            admin_username:
                azureuser
            computer_name:
                testazurevol
            linux_configuration:
                ----------
                disable_password_authentication:
                    False
            secrets:
        private_ips:
        provisioning_state:
            Succeeded
        public_ips:
        resource_group:
            AzureARM-SanRamon-New-ResGrp
        size:
            Basic_A1
        state:
            Succeeded
        storage_profile:
            ----------
            data_disks:
            image_reference:
                ----------
                offer:
                    RHEL
                publisher:
                    RedHat
                sku:
                    7.2
                version:
                    latest
            os_disk:
                ----------
                caching:
                    ReadWrite
                create_option:
                    FromImage
                disk_size_gb:
                    32
                managed_disk:
                    ----------
                    id:
                        /subscriptions/******************************8/resourceGroups/AzureARM-SanRamon-New-ResGrp/providers/Microsoft.Compute/disks/testazurevol_OsDisk_1_26d3d6471a9547f9bb72af1b3fb2ba9a
                    storage_account_type:
                        Standard_LRS
                name:
                    testazurevol_OsDisk_1_26d3d6471a9547f9bb72af1b3fb2ba9a
                os_type:
                    Linux
        type:
            Microsoft.Compute/virtualMachines
        vm_id:
            8753a40f-c73d-44c2-b8d4-c7f938484d4b

Works with older module .

[nicholasmhughes]

Can you post pip freeze | grep azure ?

[nkv16786]

On behalf of @sumeetisp , here is the output of pip freeze | grep azure

pip freeze | grep azure
azure==2.0.0rc5
azure-batch==0.30.0rc5
azure-common==1.1.4
azure-datalake-store==0.0.15
azure-graphrbac==0.30.0rc5
azure-keyvault==0.3.5
azure-mgmt==0.30.0rc5
azure-mgmt-authorization==0.30.0rc5
azure-mgmt-batch==0.30.0rc5
azure-mgmt-cdn==0.30.0rc5
azure-mgmt-cognitiveservices==0.30.0rc5
azure-mgmt-commerce==0.30.0rc5
azure-mgmt-compute==3.0.1
azure-mgmt-containerregistry==0.2.1
azure-mgmt-datalake-analytics==0.1.6
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.1.6
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==1.0.1
azure-mgmt-documentdb==0.1.3
azure-mgmt-iothub==0.2.2
azure-mgmt-keyvault==0.30.0rc5
azure-mgmt-logic==0.30.0rc5
azure-mgmt-monitor==0.2.1
azure-mgmt-network==0.30.0rc5
azure-mgmt-notificationhubs==0.30.0rc5
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==0.30.0rc5
azure-mgmt-rdbms==0.1.0
azure-mgmt-redis==0.30.0rc5
azure-mgmt-resource==0.30.0rc5
azure-mgmt-scheduler==0.30.0rc5
azure-mgmt-sql==0.5.3
azure-mgmt-storage==0.30.0rc5
azure-mgmt-trafficmanager==0.30.0
azure-mgmt-web==0.30.0rc5
azure-nspkg==2.0.0
azure-servicebus==0.20.2
azure-servicefabric==5.6.130
azure-servicemanagement-legacy==0.20.3
azure-storage==0.32.0
msrestazure==0.4.21
simpleazure==0.1.5
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

[nicholasmhughes]

I'm running much newer module versions right now, but I tested with rc6... which I believe was a dependency previous to my changes. I'll check it out later today, but updating the Azure SDK might help.

[nicholasmhughes]

Verified the IPs output as expected with:

azure==2.0.0
azure-batch==3.0.0
azure-common==1.1.8
azure-datalake-store==0.0.17
azure-graphrbac==0.30.0
azure-keyvault==0.3.7
azure-mgmt==1.0.0
azure-mgmt-authorization==0.30.0
azure-mgmt-batch==4.0.0
azure-mgmt-cdn==0.30.3
azure-mgmt-cognitiveservices==1.0.0
azure-mgmt-compute==1.0.0
azure-mgmt-containerregistry==0.2.1
azure-mgmt-datalake-analytics==0.1.6
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.1.6
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==1.0.1
azure-mgmt-documentdb==0.1.3
azure-mgmt-iothub==0.2.2
azure-mgmt-keyvault==0.31.0
azure-mgmt-logic==2.1.0
azure-mgmt-monitor==0.2.1
azure-mgmt-network==1.7.1
azure-mgmt-nspkg==2.0.0
azure-mgmt-rdbms==0.1.0
azure-mgmt-redis==4.1.1
azure-mgmt-resource==1.1.0
azure-mgmt-scheduler==1.1.3
azure-mgmt-sql==0.5.3
azure-mgmt-storage==1.0.0
azure-mgmt-trafficmanager==0.30.0
azure-mgmt-web==0.32.0
azure-nspkg==2.0.0
azure-servicebus==0.21.1
azure-servicefabric==5.6.130
azure-servicemanagement-legacy==0.20.6
azure-storage==0.34.3
msrestazure==0.4.19

I'm going to roll back and verify that's the issue.

[nicholasmhughes]

@nkv16786 I couldn't get the cloud module to run properly with the Azure SDK versions you provided. Here are the minimum versions I got to work:

azure==2.0.0rc6
azure-batch==1.0.0
azure-common==1.1.4
azure-datalake-store==0.0.15
azure-graphrbac==0.30.0rc6
azure-keyvault==0.3.5
azure-mgmt==0.30.0rc6
azure-mgmt-authorization==0.30.0rc6
azure-mgmt-batch==1.0.0
azure-mgmt-cdn==0.30.0rc6
azure-mgmt-cognitiveservices==0.30.0rc6
azure-mgmt-commerce==0.30.0rc6
azure-mgmt-compute==3.0.1
azure-mgmt-containerregistry==0.2.1
azure-mgmt-datalake-analytics==0.1.6
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.1.6
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==1.0.1
azure-mgmt-documentdb==0.1.3
azure-mgmt-iothub==0.2.2
azure-mgmt-keyvault==0.30.0rc6
azure-mgmt-logic==1.0.0
azure-mgmt-monitor==0.2.1
azure-mgmt-network==0.30.0rc6
azure-mgmt-notificationhubs==0.30.0rc6
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==0.30.0rc6
azure-mgmt-rdbms==0.1.0
azure-mgmt-redis==1.0.0
azure-mgmt-resource==0.30.0
azure-mgmt-scheduler==1.0.0
azure-mgmt-sql==0.5.3
azure-mgmt-storage==0.30.0rc6
azure-mgmt-trafficmanager==0.30.0
azure-mgmt-web==0.30.0rc6
azure-nspkg==2.0.0
azure-servicebus==0.20.2
azure-servicefabric==5.6.130
azure-servicemanagement-legacy==0.20.3
azure-storage==0.32.0
msrestazure==0.4.21

...although I'd recommend upgrading further since I'm planning on committing some execution and state modules to create other infrastructure (resource groups, virtual networks, availability sets, etc.) in Azure, and the required Azure SDK and module versions will be higher.

This was tested on a brand new CentOS 7 server running 2017.7.3, only 3 extra packages installed (python2-pip, python-devel, and gcc), and the reqs above installed with pip.

[sumeetisp]

I have upgraded the libraries but now provisioning is failing.
NIC card is getting provisioned but VM is not getting provisioned which was happening earlier with old libraries.
I see following issues,
[DEBUG ] Failed to get data for node 'test-vol4'
[DEBUG ] 'update_callback' has returned 'False', which is considered a failure. Remaining Failures: 10.
Which is obvious because the vm itself was not provisioned

[sumeetisp]

This what i have in my env,

azure==2.0.0rc6
azure-batch==1.0.0
azure-common==1.1.4
azure-datalake-store==0.0.15
azure-graphrbac==0.30.0rc6
azure-keyvault==0.3.5
azure-mgmt==0.30.0rc6
azure-mgmt-authorization==0.30.0rc6
azure-mgmt-batch==1.0.0
azure-mgmt-cdn==0.30.0rc6
azure-mgmt-cognitiveservices==0.30.0rc6
azure-mgmt-commerce==0.30.0rc6
azure-mgmt-compute==0.30.0rc6
azure-mgmt-containerregistry==0.2.1
azure-mgmt-datalake-analytics==0.1.6
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.1.6
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==1.0.1
azure-mgmt-documentdb==0.1.3
azure-mgmt-iothub==0.2.2
azure-mgmt-keyvault==0.30.0rc6
azure-mgmt-logic==1.0.0
azure-mgmt-monitor==0.2.1
azure-mgmt-network==0.30.0rc6
azure-mgmt-notificationhubs==0.30.0rc5
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==0.30.0rc5
azure-mgmt-rdbms==0.1.0
azure-mgmt-redis==1.0.0
azure-mgmt-resource==0.30.0rc6
azure-mgmt-scheduler==1.0.0
azure-mgmt-sql==0.5.3
azure-mgmt-storage==0.30.0rc6
azure-mgmt-trafficmanager==0.30.0
azure-mgmt-web==0.30.0rc6
azure-nspkg==2.0.0
azure-servicebus==0.20.3
azure-servicefabric==5.6.130
azure-servicemanagement-legacy==0.20.4
azure-storage==0.33.0
azure-storage-blob==1.1.0
azure-storage-common==1.1.0
azure-storage-nspkg==3.0.0
msrestazure==0.4.21
simpleazure==0.1.5

[nicholasmhughes]

Try with these minimum versions:

azure-mgmt-compute==0.33.0
azure-mgmt-resource==0.30.0

If this works, I'll update the documentation to include the minimum versions for each module instead of the overall SDK.

Also, I found a bug where the OS disk sizing isn't working with the managed disks. I'll file a PR with that fix independently or with the larger set of modules I mentioned once we get this sorted out.

[sumeetisp]

I will try this and revert on Monday. Earlier the only issue was the output
but the VM was successful provisioned.
But now the interface gets provisioned and vm doesn't.

Thanks,
Sumeet Salvankar

On 16-Feb-2018 19:39, "Nicholas Hughes" [e-mail] wrote:

Try with these minimum versions:

azure-mgmt-compute==0.33.0
azure-mgmt-resource==0.30.0

If this works, I'll update the documentation to include the minimum
versions for each module instead of of the overall SDK.

Also, I found a bug where the OS disk sizing isn't working with the managed
disks. I'll file a PR with that fix independently or with the larger set of
modules I mentioned once we get this sorted out.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-366244754,
or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHg-E23SuQ-zKn2CwxX6Zse_gWl_PFGhks5tVYwWgaJpZM4Q7FX5
.

[sumeetisp]

It worked.
Please elaborate a bit on the issue you have found on os disk sizing of managed disk. Is it provisioning with the size size of the image and not the value passed in os_disk_size_gb?

[nicholasmhughes]

Correct. That value isn't passed properly.

[sumeetisp]

One question for code at https://github.com/saltstack/salt/blob/develop/salt/cloud/clouds/azurearm.py#L1325

Do we need this even when deploy is set to false?

Also can we name it as ssh_interface so that it is common across providers. Currently it is ssh_interface for aws.

[nicholasmhughes]

That logic needs to be reworked. There is another issue (I'm on my phone, otherwise I'd cite the number) that notes it's not possible to create dynamic public IP addresses. With the current logic, creating a dynamic public IP would result in waiting a really long time for the IP to come up... but it won't until the VM actually boots. That will also need to be changed.

Regarding the parameter naming, I'm not terribly fond of the AWS nomenclature. Additionally, I wouldn't want to break people's existing provider files. Maybe we accommodate both.

I'll take a look at fixing some of this soon.

[sumeetisp]

Hi @nicholasmhughes , Yesterday provisioning was working perfectly with proper output. But today provisioning i working but i dont get output as expected. Also show_instance and stop action are not able to find provisioned machines.

[sumeetisp]

Restarting the master helped. Weird :D

[sumeetisp]

We tried with windows image which had an os disk of around 150gb, failed
saying "failed waiting for an ip" ( something like this). But link which
has 30gb os disk provisioned successfully.

I guess this is the issue you were mentioning.

Thanks,
Sumeet Salvankar

On 19-Feb-2018 18:44, "Nicholas Hughes" [e-mail] wrote:

That logic needs to be reworked. There is another issue (I'm on my phone,
otherwise I'd cite the number) that notes it's not possible to create
dynamic public IP addresses. With the current logic, creating a dynamic
public IP would result in waiting a really long time for the IP to come
up... but it won't until the VM actually boots. That will also need to be
changed.

Regarding the parameter naming, I'm not terribly fond of the AWS
nomenclature. Additionally, I wouldn't want to break people's existing
provider files. Maybe we accommodate both.

I'll take a look at fixing some of this soon.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-366689714,
or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHg-EzwdBWNDYvidTmctSjb47a4isRqqks5tWXOsgaJpZM4Q7FX5
.

[nicholasmhughes]

@sumeetisp , this section:

https://github.com/saltstack/salt/pull/46186/files#diff-37c85ff409208fd9e1bbb7eb7baaa96cR1199

of PR #46186 fixes the OS disk sizing. It would be pretty easy to hack that 4 line change into your server if you need that support right now. I still haven't messed with reworking the IP waits or other logic for deployment... but there are quite a few Azure infrastructure execution and state modules in that PR which might interest you.

[sumeetisp]

Hi @nicholasmhughes , In previous versions,
expire_publisher_cache: 604800 # 7 days
expire_offer_cache: 518400 # 6 days
expire_sku_cache: 432000 # 5 days
expire_version_cache: 345600 # 4 days
expire_group_cache: 14400 # 4 hours
expire_interface_cache: 3600 # 1 hour
expire_network_cache: 3600 # 1 hour

This was also an issue. Because if someone would delete directly from azure portal then cache data would throw error.

[sumeetisp]

Hi @nicholasmhughes ,

Is this code at https://github.com/nicholasmhughes/salt/blob/c50c44cec4b15eddf7ed44a8e1ab1af38093cdf2/salt/cloud/clouds/azurearm.py#L1176
required even when deploy is set false?

VM with managed disk went through but with unmanaged disk got following error,
[DEBUG ] {
"error": {
"code": "InvalidParameter",
"message": "The value 'Linux' of parameter 'osDisk.osType' is not allowed. Allowed values are 'Windows'."
}
}
[DEBUG ] The value 'Linux' of parameter 'osDisk.osType' is not allowed. Allowed values are 'Windows'.
[ERROR ] An AzureARM Compute CloudError has occurred: The value 'Linux' of parameter 'osDisk.osType' is not allowed. Allowed values are 'Windows'.
[ERROR ] Error creating VM test-disk1! ({})
Error: There was a profile error: Error creating VM test-disk1! ({})

[nicholasmhughes]

Again... I haven't had a chance to rework the deploy logic. I'm sure there are a lot of things that can be optimized.

However, the error you're seeing looks like a SDK version problem. Try bumping the azure-mgmt-compute version until the error goes away... I would do it myself, but I don't think I'll have time this week.

[sumeetisp]

If i upgrade the SDK version. It breaks imports ,
File "/tmp/test.py", line 27, in
import azure.mgmt.compute.models as compute_models
ImportError: No module named azure.mgmt.compute.models.

[nicholasmhughes]

There is often some weird behavior when upgrading. Uninstalling the module and then reinstalling fresh should help.

[nkv16786]

Hi,
I pulled recent azurearm code from develop branch and did show_instance got the following error:-

[DEBUG ] Failed to execute 'azurearm.list_nodes()' while querying for running nodes: u'image_reference'
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 2396, in run_parallel_map_providers_query
cloud.cloudsdata['fun']
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 490, in list_nodes
nodes = list_nodes_full()
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 577, in list_nodes_full
group_ret = {k: v for result in results.get() for k, v in six.iteritems(result)}
File "/usr/lib/python2.7/multiprocessing/pool.py", line 558, in get
raise self._value
KeyError: u'image_reference'

[nicholasmhughes]

@nkv16786 what azure sdk versions are you running?

[sumeetisp]

He has updated on my behalf. So ask version has not be modified after I
confirmed earlier.

Thanks,
Sumeet Salvankar

On 09-Mar-2018 22:13, "Nicholas Hughes" [e-mail] wrote:

@nkv16786 https://github.com/nkv16786 what azure sdk versions are you
running?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/44886#issuecomment-371867804,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHg-Exj2Yi7AOsvHl1GsFtmXn3jM8oO8ks5tcrE5gaJpZM4Q7FX5
.

[nicholasmhughes]

Ok. During the last exchange, I asked if you could upgrade your SDK versions. You noted that "it broke your imports", and I said that I've seen that before... and that removal and fresh installation via pip should fix it.

If you haven't upgraded your module versions, then that needs to be done.

If you have upgraded your module versions, then I'd like to see the versions.

[bsipswitch]

I seem to be bumping into this same issue.

[DEBUG   ] Failed to execute 'azurearm.list_nodes()' while querying for running nodes: u'image_reference'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 2393, in run_parallel_map_providers_query
    cloud.clouds[data['fun']]()
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/azurearm.py", line 490, in list_nodes
    nodes = list_nodes_full()
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/azurearm.py", line 577, in list_nodes_full
    group_ret = {k: v for result in results.get() for k, v in six.iteritems(result)}
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
KeyError: u'image_reference'
[DEBUG   ] LazyLoaded nested.output
Not Actioned/Not Running:
    - somevm
Not Found:
    - somevm

My module versions:

azure==3.0.0
azure-batch==4.0.0
azure-common==1.1.8
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.2
azure-datalake-store==0.0.19
azure-eventgrid==0.1.0
azure-graphrbac==0.40.0
azure-keyvault==0.3.7
azure-mgmt==2.0.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.30.0
azure-mgmt-batch==5.0.0
azure-mgmt-batchai==0.2.0
azure-mgmt-billing==0.1.0
azure-mgmt-cdn==2.0.0
azure-mgmt-cognitiveservices==2.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==3.1.0rc3
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==0.3.1
azure-mgmt-containerregistry==1.0.1
azure-mgmt-containerservice==3.0.1
azure-mgmt-cosmosdb==0.3.1
azure-mgmt-datafactory==0.4.0
azure-mgmt-datalake-analytics==0.3.0
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.3.0
azure-mgmt-devtestlabs==2.2.0
azure-mgmt-dns==1.2.0
azure-mgmt-eventgrid==0.4.0
azure-mgmt-eventhub==1.2.0
azure-mgmt-hanaonazure==0.1.0
azure-mgmt-iothub==0.4.0
azure-mgmt-iothubprovisioningservices==0.1.0
azure-mgmt-keyvault==0.40.0
azure-mgmt-loganalytics==0.1.0
azure-mgmt-logic==2.1.0
azure-mgmt-machinelearningcompute==0.4.0
azure-mgmt-managementpartner==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==0.2.0
azure-mgmt-monitor==0.4.0
azure-mgmt-msi==0.1.0
azure-mgmt-network==1.7.1
azure-mgmt-notificationhubs==1.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==1.0.0
azure-mgmt-rdbms==0.1.0
azure-mgmt-recoveryservices==0.2.0
azure-mgmt-recoveryservicesbackup==0.1.1
azure-mgmt-redis==5.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.1.0
azure-mgmt-resource==1.2.2
azure-mgmt-scheduler==1.1.3
azure-mgmt-search==1.0.0
azure-mgmt-servermanager==1.2.0
azure-mgmt-servicebus==0.4.0
azure-mgmt-servicefabric==0.1.0
azure-mgmt-sql==0.8.5
azure-mgmt-storage==1.5.0
azure-mgmt-subscription==0.1.0
azure-mgmt-trafficmanager==0.40.0
azure-mgmt-web==0.34.1
azure-nspkg==2.0.0
azure-servicebus==0.21.1
azure-servicefabric==6.1.2.9
azure-servicemanagement-legacy==0.20.6
azure-storage-blob==1.1.0
azure-storage-common==1.1.0
azure-storage-file==1.1.0
azure-storage-nspkg==3.0.0
azure-storage-queue==1.1.0
msrestazure==0.4.22

Am i using the wrong versions?

[nicholasmhughes]

@bsipswitch those versions look new. I'll see if I can test that out. Thanks for providing the pip freeze output!

[nicholasmhughes]

Ok... the stack trace line number is a little misleading due to multi. There's a lot of static stuff here:

https://github.com/saltstack/salt/blob/develop/salt/cloud/clouds/azurearm.py#L529

... that I've been wanting to change. It's making assumptions about the return data from the node query. I'll see what I can do.

[nicholasmhughes]

@bsipswitch I think the wrong exception is being caught (TypeError instead of KeyError). Can you make this change and let me know if it's the fix?

Old:

   536          except TypeError:
   537              try:
   538                  node['image'] = node['storage_profile']['os_disk']['image']['uri']
   539              except TypeError:
   540                  node['image'] = None

New:

   536          except KeyError:
   537              try:
   538                  node['image'] = node['storage_profile']['os_disk']['image']['uri']
   539              except KeyError:
   540                  node['image'] = None

[bsipswitch]

With that change:

salt-cloud -a show_instance somevm - works
salt-cloud -f list_nodes myconfig - also works

I did a test vm creation after words and I get this:

salt-cloud -p someprof somevm -l debug

[ERROR   ] There was a profile error: list index out of range
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/cloud/cli.py", line 281, in run
    self.config.get('names')
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1454, in run_profile
    ret[name] = self.create(vm_)
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1284, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/azurearm.py", line 1408, in create
    'wait_for_ip_interval_multiplier', vm_, __opts__, default=1),
  File "/usr/lib/python2.7/site-packages/salt/utils/cloud.py", line 2389, in wait_for_ip
    data = update_callback(*update_args, **update_kwargs)
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/azurearm.py", line 1392, in _query_node_data
    ip_address = data['public_ips'][0]
IndexError: list index out of range

But I think that exception change fixed the original issue.

I'm not sure about the index out of range issue.

If i set
public_ip: True
in the profile the interface is not created with a public ip.
If i set it to create the vm without the public IP i get the same issue.

Looks like from the rolling debug logs its querying all the interfaces in subscription and not finding the public ip. Even if the vm created is slated only to have a private ip.

I could be missing the obvious though.

[nicholasmhughes]

Ok, so PR #46625 should fix the KeyError issue in develop once it's merged.

That IndexError you're seeing is interesting... It sounds like you're not seeing a public IP created at all.

To be sure, the parameter to cause a public IP to be created is: allocate_public_ip: True

If a public IP isn't created, and the bootstrap_interface parameter is set to public (or isn't set, so it defaults to public), then the code looks for a public IP address in the VM return data that won't have one... and fails pretty abruptly. The workaround is to set bootstrap_interface: private for anything that won't have a public IP and ensure that allocate_public_ip: True is set for anything that will... That should work until the code is refactored to handle this better.

Let me know if that works for you.

[bsipswitch]

bootstrap_interface: private
works for things with only private ip addresses
and
bootstrap_interface: private
allocate_public_ip: True
for things with public and private,
great stuff both work.

I think i tried these before when the original error was happening with no success, but now with the KeyError fix all works correctly.

thanks man

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.