Salt: Profile Error when trying to generate an Azure VM from GCP

[justin-w-long]

Description of Issue/Question

My Salt Master is in GCP and I am trying to get vm generation in Azure to work from a GCP master. I am able to communicate with Azure and it begins the process of spinning up the VM but it fails during the creation with a profile error.

Setup

Provider Configuration:

azure-ubuntu:
  driver: azurearm
  subscription_id: SUBSCRITION_ID

  tenant: TENANT_ID
  client_id: CLIENT_ID
  secret: MY_SECRET

  location: eastus
  resource_group: AzureDemo
  network_resource_group: AzureDemo
  storage_account: MY_STORAGE_ACCOUNT
  storage_key: MY_STORAGE_KEY

  cleanup_disks: True
  cleanup_vhds: True
  cleanup_data_disks: True
  cleanup_interfaces: True

  # Set up the location of the salt master
  #
  minion:
    master: MASTER_IP

Profile Configuration:

azure-ubuntu:
  provider: azure-ubuntu
  image: Canonical|UbuntuServer|18.04-LTS|18.04.201804262
  size: Standard_DS1
  ssh_username: MY_USERNAME
  ssh_password: MY_PASSWORD
  location: eastus

  network: AzureDemo-Network
  public_ip: False

Steps to Reproduce Issue

Run the command: "salt-cloud -l debug -p azure-ubuntu test-demo"

[ERROR ] There was a profile error: __init__() takes exactly 1 argument (2 given)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 281, in run
self.config.get('names')
File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1454, in run_profile
ret[name] = self.create(vm_)
File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1284, in create
output = self.cloudsfunc
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1279, in create
'wait_for_ip_interval_multiplier', vm_, __opts__, default=1),
File "/usr/lib/python2.7/dist-packages/salt/utils/cloud.py", line 2389, in wait_for_ip
data = update_callback(update_args, *update_kwargs)
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1254, in _query_ip_address
data = request_instance(kwargs=vm_)
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1052, in request_instance
iface_data = create_interface(kwargs=vm_)
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 926, in create_interface
six.text_type(pub_ip_data.id), # pylint: disable=no-member
TypeError: __init__() takes exactly 1 argument (2 given)

Versions Report

Salt Version:
Salt: 2018.3.0

Dependency Versions:
cffi: 1.11.5
cherrypy: 3.5.0
dateutil: 2.7.2
docker-py: Not Installed
gitdb: 0.6.4
gitpython: 1.0.1
ioflo: Not Installed
Jinja2: 2.8
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: 1.0.3
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: 2.18
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.12 (default, Dec 4 2017, 14:50:18)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.2.0
RAET: Not Installed
smmap: 0.9.0
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4

System Versions:
dist: Ubuntu 16.04 xenial
locale: UTF-8
machine: x86_64
release: 4.4.0-72-generic
system: Linux
version: Ubuntu 16.04 xenial

[gtmanfred]

@nicholasmhughes can you help me with this one?

Thanks,
Daniel

[nicholasmhughes]

@justin-w-long can you post the output of pip freeze | egrep 'azure|msrest' on the master?

[justin-w-long]

azure==3.0.0
azure-batch==4.0.0
azure-batch-extensions==1.0.1
azure-cli==2.0.30
azure-cli-acr==2.0.22
azure-cli-acs==2.0.30
azure-cli-advisor==0.5.0
azure-cli-appservice==0.1.30
azure-cli-backup==1.1.0
azure-cli-batch==3.1.11
azure-cli-batchai==0.1.6
azure-cli-billing==0.1.7
azure-cli-cdn==0.0.13
azure-cli-cloud==2.0.12
azure-cli-cognitiveservices==0.1.11
azure-cli-command-modules-nspkg==2.0.1
azure-cli-configure==2.0.14
azure-cli-consumption==0.2.2
azure-cli-container==0.1.20
azure-cli-core==2.0.30
azure-cli-cosmosdb==0.1.19
azure-cli-dla==0.0.18
azure-cli-dls==0.0.20
azure-cli-eventgrid==0.1.11
azure-cli-eventhubs==0.1.1
azure-cli-extension==0.0.11
azure-cli-feedback==2.1.0
azure-cli-find==0.2.8
azure-cli-interactive==0.3.18
azure-cli-iot==0.1.18
azure-cli-keyvault==2.0.20
azure-cli-lab==0.0.20
azure-cli-monitor==0.1.4
azure-cli-network==2.0.26
azure-cli-nspkg==3.0.2
azure-cli-profile==2.0.21
azure-cli-rdbms==0.2.0
azure-cli-redis==0.2.11
azure-cli-reservations==0.1.1
azure-cli-resource==2.0.26
azure-cli-role==2.0.21
azure-cli-servicebus==0.1.1
azure-cli-servicefabric==0.0.11
azure-cli-sql==2.0.24
azure-cli-storage==2.0.30
azure-cli-vm==2.0.29
azure-common==1.1.8
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.2
azure-datalake-store==0.0.19
azure-eventgrid==0.1.0
azure-graphrbac==0.40.0
azure-keyvault==0.3.7
azure-mgmt==2.0.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.40.0
azure-mgmt-batch==4.1.0
azure-mgmt-batchai==0.2.0
azure-mgmt-billing==0.1.0
azure-mgmt-cdn==1.0.0
azure-mgmt-cognitiveservices==1.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==4.0.0rc1
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==0.4.0
azure-mgmt-containerregistry==1.0.1
azure-mgmt-containerservice==3.0.1
azure-mgmt-cosmosdb==0.3.1
azure-mgmt-datafactory==0.4.0
azure-mgmt-datalake-analytics==0.2.0
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.2.0
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==2.0.0rc1
azure-mgmt-eventgrid==0.4.0
azure-mgmt-eventhub==1.2.0
azure-mgmt-hanaonazure==0.1.0
azure-mgmt-iothub==0.4.0
azure-mgmt-iothubprovisioningservices==0.1.0
azure-mgmt-keyvault==0.40.0
azure-mgmt-loganalytics==0.1.0
azure-mgmt-logic==2.1.0
azure-mgmt-machinelearningcompute==0.4.0
azure-mgmt-managementpartner==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==0.2.0
azure-mgmt-monitor==0.5.0
azure-mgmt-msi==0.1.0
azure-mgmt-network==2.0.0rc1
azure-mgmt-notificationhubs==1.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==1.0.0
azure-mgmt-rdbms==1.0.0
azure-mgmt-recoveryservices==0.1.0
azure-mgmt-recoveryservicesbackup==0.1.1
azure-mgmt-redis==4.1.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.1.0
azure-mgmt-resource==1.2.1
azure-mgmt-scheduler==1.1.3
azure-mgmt-search==1.0.0
azure-mgmt-servermanager==1.2.0
azure-mgmt-servicebus==0.4.0
azure-mgmt-servicefabric==0.1.0
azure-mgmt-sql==0.8.5
azure-mgmt-storage==1.5.0
azure-mgmt-subscription==0.1.0
azure-mgmt-trafficmanager==0.40.0
azure-mgmt-web==0.35.0
azure-multiapi-storage==0.2.0
azure-nspkg==2.0.0
azure-servicebus==0.21.1
azure-servicefabric==6.1.2.9
azure-servicemanagement-legacy==0.20.6
azure-storage==0.34.3
azure-storage-blob==1.1.0
azure-storage-common==1.1.0
azure-storage-file==1.1.0
azure-storage-nspkg==3.0.0
azure-storage-queue==1.1.0
You are using pip version 8.1.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
msrest==0.4.27
msrestazure==0.4.25

[nicholasmhughes]

@justin-w-long thanks! It looks like you're failing on a section of code that should only execute when public_ip: True... so, that's weird. I tried it with the same options and got past that section of code. I subsequently failed because I didn't provide storage account information in my provider. You might consider using the Azure driver from develop, because it has a few nice updates and most notably includes the ability to use managed disks (the MS recommended option).

Also, I noticed that you're setting public_ip: False, but I don't see deploy: False... which will cause you problems during bootstrap. If you don't need minion software on the host, then it's no problem. Otherwise, the code in develop will allow you to pass a userdata script as a VM Extension to install Salt and point it at your master's public IP.

@gtmanfred , do you know when we are going to see the Azure code from develop hit a stable release? It looks like the code in 2018.3 and develop are diverging...

[justin-w-long]

It looks like it is still generating a public IP address in the Azure Console even though the value is set to False.

[nicholasmhughes]

Yeah... I can't explain that. It's not happening to me. Make sure there aren't any weird characters in your profile definition file. It could cause that parameter to get missed.

In other news, I reproduced a similar TypeError to what you were seeing. It seems that newer versions of the Azure SDK don't like ambiguity in their object models.

This failed:

                    ip_kwargs['public_ip_address'] = PublicIPAddress(
                        six.text_type(pub_ip_data.id),  # pylint: disable=no-member
)

...but adding the explicit id= worked:

                    ip_kwargs['public_ip_address'] = PublicIPAddress(
                        id=six.text_type(pub_ip_data.id),  # pylint: disable=no-member
)

Also, I found some other "interesting" behavior in the newer version of the Azure SDK surrounding strings vs. unicode.

From /usr/lib/python2.7/site-packages/azure/mgmt/resource/resources/resource_management_client.py:

        if not isinstance(subscription_id, str):
            raise TypeError("Parameter 'subscription_id' must be str.")

...that's just sloppy.

The short version is that you might have to downgrade your Azure SDK versions until things "just work".

[justin-w-long]

So I realized I had another profile that I was testing that had the same profile name causing the public IP to still be spun up. After removing that other profile it makes it pass the portion of the code for publicIP, but throws the same error not long after when interacting with disk name and storage information.

[ERROR ] There was a profile error: __init__() takes exactly 1 argument (2 given)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 281, in run
self.config.get('names')
File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1454, in run_profile
ret[name] = self.create(vm_)
File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1284, in create
output = self.cloudsfunc
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1279, in create
'wait_for_ip_interval_multiplier', vm_, __opts__, default=1),
File "/usr/lib/python2.7/dist-packages/salt/utils/cloud.py", line 2389, in wait_for_ip
data = update_callback(update_args, *update_kwargs)
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1254, in _query_ip_address
data = request_instance(kwargs=vm_)
File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1177, in request_instance
disk_name,
TypeError: __init__() takes exactly 1 argument (2 given)

[nicholasmhughes]

I think your best best is downgrading these Azure SDK modules:

azure-mgmt-compute==3.1.0rc3
azure-mgmt-network==1.7.1

I had success creating an instance with the versions I listed above and the latest 2018.3 code from Git which includes the logic to ensure subscription_id is a string.

[justin-w-long]

I am now able to get a virtual machine to generate but it fails because it says my subscription ID is not a string. I am on version 2018.3 according to salt --version, do I need to reinstall it or something to get the logic to make sure my subscription_id is a string?

[nicholasmhughes]

It's in the 2018.3 branch, which is where patch releases will get cut. It's pretty safe to pull it from there.

[justin-w-long]

After getting the latest 2018.3 branch code and rolling back those versions of azure-mgmt-compute and network I am able to spin up VM's now. Thank you for all your help!

[gtmanfred]

Thanks for the help @nicholasmhughes

It is greatly appreciated!

Daniel