Terraform: azurerm - error creating vm from existing vhd - osType not supplied

[ghost]

Terraform Version

v0.6.15

Affected Resource(s)

azurerm_virtual_machine

Terraform Configuration Files

resource "azurerm_virtual_machine" "ap-service-discovery-server-0" {
    name = "ap-service-discovery-server-0"
    resource_group_name = "${var.resource_group_name}"
    location = "${var.location}"
    vm_size = "${var.instance_type}"
    network_interface_ids = ["${azurerm_network_interface.ap-service-discovery-server-0.id}"]
    availability_set_id = "${azurerm_availability_set.ap-service-discovery-availability-set.id}"

    #TODO at this point - copy from vhd storage, rename and move to VM storage acct.
    storage_os_disk {
        name = "ap-service-discovery-server-0-osdisk"
        vhd_uri = "${replace("${var.vhd_path}",".vhd","-0.vhd")}"
        create_option = "attach"
    }

    os_profile {
        computer_name = "ap-service-discovery-server-0"
        admin_username = "${var.ssh_username}"
        admin_password = "${var.ssh_password}"
        custom_data = "${template_file.user_data_0.rendered}"
    }

    os_profile_linux_config {
       disable_password_authentication = false
        /*ssh_keys {
          path = "/home/${var.ssh_username}/.ssh/authorized_keys"
          key_data = "${file("${var.ssh_key_path}")}"
        }*/
    }
}

Error

* azurerm_virtual_machine.ap-service-discovery-server-0: autorest:DoErrorUnlessStatusCode 400 PUT https://management.azure.com/subscriptions/8b086f2c-3ade-4770-8825-d97490e3fe83/resourceGroups/core-dev/providers/Microsoft.Compute/virtualMachines/ap-service-discovery-server-0?api-version=2015-06-15 failed with 400 Bad Request

from charles proxy:

"properties": {
            "statusCode": "BadRequest",
            "serviceRequestId": "2ab50479-ef29-4b24-95fe-8fd8967831cd",
            "statusMessage": "{\"error\":{\"code\":\"InvalidParameter\",\"target\":\"osDisk.osType\",\"message\":\"Required parameter 'osDisk.osType' is missing (null).\"}}"
        },

Expected Behavior

What should have happened?
The VM should have successfully been created with the specified OS disk.

osType is not mentioned in the example for the virtual machine resource and no mapping seems to exist.

Actual Behaviour

error 400 bad request - see charles output above.

[stack72]

Hi @tasquith

thanks for reporting the error. I am currently looking into the API for the os_type parameter. Can you tell me how you created the os_disk? This would allow me to try and create one as a test

Paul

[stack72]

@tasquith

Just a FYI, the reason this wasn't added (os_type) is because it hasn't been documented on the Azure API - https://msdn.microsoft.com/en-us/library/mt163591.aspx#bk_osdisk

I will try and debug what is needed

Thanks again for pointing this out as it is a showstopper

P.

[ghost]

Hi @stack72,

Thanks for getting back to me so quickly!

It should just work using packer to create an image based on the openlogic 7 template.

as a side note, I have to create it using packer with the classic provider (we create a base image from openlogic, then use that to create a more specialised server - this one, I have to do this as the azurerm provider for packer doesn't yet support using a vhd as a seed to create another image), then use the cli to move it to a rm storage account, where I have to copy it x number of times using the cli , before using terraform azurerm to create x number of nodes. Is there/will there be a way of using a VHD as a source in terraform, where it then creates its own copy in a specified storage acct for each instance needed, rather than just attaching?

If you need anything more, just let me know.

Thanks,

Tom

[stack72]

@tasquith

This is _REALLY USEFUL_ info! This will help me create a good test case here - so thanks for that. Let me see if i can fix this bug and then see if we can address the second part

Paul

[stack72]

Hi @tasquith

I am currently tracking this issue now as a larger set of AzureRM work. I am going to close this issue so that everything can be tracked on #6526

Paul

[stack72]

Hi @tasquith

Are you in a position to be able to build terraform from source to run a test for this issue? A PR has been very helpfully created by @Erouan50 and I'd like to try and include it in 0.6.16

Paul

[ghost]

Hi @stack72,

I'll take a look now - progress may be slow as I've not built it before but it should be fine!

Tom

[ghost]

Hi @stack72,

I built the provider successfully, but I couldn't get past plan - so I assume the fix is in the core?

I tried building the whole lot just using make - but got the following issue:

ok github.com/hashicorp/terraform 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-atlas 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-aws 0.022s ok github.com/hashicorp/terraform/builtin/bins/provider-azure 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-azurerm [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-chef [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-clc 0.018s ok github.com/hashicorp/terraform/builtin/bins/provider-cloudflare 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-cloudstack 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-cobbler [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-consul 0.034s ok github.com/hashicorp/terraform/builtin/bins/provider-datadog 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-digitalocean 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dme 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dnsimple 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-docker 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dyn 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-fastly 0.018s ok github.com/hashicorp/terraform/builtin/bins/provider-github 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-google 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-heroku 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-influxdb 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-librato 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-mailgun 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-mysql 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-null 0.018s ? github.com/hashicorp/terraform/builtin/bins/provider-openstack [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-packet [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-postgresql 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-powerdns 0.019s ? github.com/hashicorp/terraform/builtin/bins/provider-rundeck [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-softlayer 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-statuscake [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-template [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-terraform 0.021s ? github.com/hashicorp/terraform/builtin/bins/provider-tls [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-triton [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-ultradns [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-vcd [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-vsphere 0.028s ok github.com/hashicorp/terraform/builtin/bins/provisioner-chef 0.020s ok github.com/hashicorp/terraform/builtin/bins/provisioner-file 0.021s ok github.com/hashicorp/terraform/builtin/bins/provisioner-local-exec 0.019s ok github.com/hashicorp/terraform/builtin/bins/provisioner-remote-exec 0.019s ok github.com/hashicorp/terraform/builtin/providers/atlas 0.018s ok github.com/hashicorp/terraform/builtin/providers/aws 1.702s ok github.com/hashicorp/terraform/builtin/providers/azure 0.032s ok github.com/hashicorp/terraform/builtin/providers/azurerm 0.027s ok github.com/hashicorp/terraform/builtin/providers/chef 0.018s ok github.com/hashicorp/terraform/builtin/providers/clc 0.018s ok github.com/hashicorp/terraform/builtin/providers/cloudflare 0.018s ok github.com/hashicorp/terraform/builtin/providers/cloudstack 0.022s ok github.com/hashicorp/terraform/builtin/providers/cobbler 0.022s ok github.com/hashicorp/terraform/builtin/providers/consul 0.020s ok github.com/hashicorp/terraform/builtin/providers/datadog 0.020s ok github.com/hashicorp/terraform/builtin/providers/digitalocean 0.019s ok github.com/hashicorp/terraform/builtin/providers/dme 0.018s ok github.com/hashicorp/terraform/builtin/providers/dnsimple 0.019s ok github.com/hashicorp/terraform/builtin/providers/docker 0.019s ok github.com/hashicorp/terraform/builtin/providers/dyn 0.018s ok github.com/hashicorp/terraform/builtin/providers/fastly 0.020s ok github.com/hashicorp/terraform/builtin/providers/github 0.019s ok github.com/hashicorp/terraform/builtin/providers/google 0.034s ok github.com/hashicorp/terraform/builtin/providers/heroku 0.022s ok github.com/hashicorp/terraform/builtin/providers/influxdb 0.020s ok github.com/hashicorp/terraform/builtin/providers/librato 0.019s ok github.com/hashicorp/terraform/builtin/providers/mailgun 0.020s ok github.com/hashicorp/terraform/builtin/providers/mysql 0.018s ok github.com/hashicorp/terraform/builtin/providers/null 0.018s ok github.com/hashicorp/terraform/builtin/providers/openstack 0.022s ok github.com/hashicorp/terraform/builtin/providers/packet 0.019s ok github.com/hashicorp/terraform/builtin/providers/postgresql 0.019s ok github.com/hashicorp/terraform/builtin/providers/powerdns 0.019s ok github.com/hashicorp/terraform/builtin/providers/rundeck 0.019s ok github.com/hashicorp/terraform/builtin/providers/softlayer 0.019s ok github.com/hashicorp/terraform/builtin/providers/statuscake 0.018s ok github.com/hashicorp/terraform/builtin/providers/template 0.019s ok github.com/hashicorp/terraform/builtin/providers/terraform 0.021s ok github.com/hashicorp/terraform/builtin/providers/test 0.212s ok github.com/hashicorp/terraform/builtin/providers/tls 0.021s ok github.com/hashicorp/terraform/builtin/providers/triton 0.021s ok github.com/hashicorp/terraform/builtin/providers/ultradns 0.018s ok github.com/hashicorp/terraform/builtin/providers/vcd 0.018s ok github.com/hashicorp/terraform/builtin/providers/vsphere 0.032s ok github.com/hashicorp/terraform/builtin/provisioners/chef 0.033s ok github.com/hashicorp/terraform/builtin/provisioners/file 0.018s ok github.com/hashicorp/terraform/builtin/provisioners/local-exec 0.054s ok github.com/hashicorp/terraform/builtin/provisioners/remote-exec 0.018s ok github.com/hashicorp/terraform/command 1.446s ok github.com/hashicorp/terraform/communicator 0.019s ok github.com/hashicorp/terraform/communicator/remote 0.011s ok github.com/hashicorp/terraform/communicator/ssh 0.056s ok github.com/hashicorp/terraform/communicator/winrm 0.034s ok github.com/hashicorp/terraform/config 0.077s ok github.com/hashicorp/terraform/config/module 0.036s ok github.com/hashicorp/terraform/dag 0.014s ok github.com/hashicorp/terraform/digraph 0.011s ? github.com/hashicorp/terraform/dot [no test files] ok github.com/hashicorp/terraform/flatmap 0.012s ? github.com/hashicorp/terraform/helper/acctest [no test files] ok github.com/hashicorp/terraform/helper/config 0.018s ok github.com/hashicorp/terraform/helper/diff 0.018s ok github.com/hashicorp/terraform/helper/hashcode 0.009s ? github.com/hashicorp/terraform/helper/logging [no test files] ok github.com/hashicorp/terraform/helper/mutexkv 0.062s ok github.com/hashicorp/terraform/helper/pathorcontents 0.014s ok github.com/hashicorp/terraform/helper/resource 15.061s ok github.com/hashicorp/terraform/helper/schema 0.023s ok github.com/hashicorp/terraform/plugin 0.259s ok github.com/hashicorp/terraform/rpc 0.038s ok github.com/hashicorp/terraform/state 0.023s 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] Handling Atlas conflict response: Conflict 2016/05/09 17:22:09 [DEBUG] Atlas reported conflict, checking for equivalent states. 2016/05/09 17:22:09 [DEBUG] GET http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] States are equivalent, incrementing serial and retrying. 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49445/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49447/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] Handling Atlas conflict response: Conflict 2016/05/09 17:22:09 [DEBUG] Atlas reported conflict, checking for equivalent states. 2016/05/09 17:22:09 [DEBUG] GET http://127.0.0.1:49447/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] States are not equivalent, returning conflict. 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= --- FAIL: TestAtlasClient_UnresolvableConflict (0.05s) atlas_test.go:163: Timed out after 50ms, probably because retrying infinitely. 2016/05/09 17:22:09 [ERR] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= request failed: Put http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id=: read tcp 127.0.0.1:49451->127.0.0.1:49450: read: connection reset by peer 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id=: retrying in 1s 2016/05/09 17:22:10 [WARN] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id FAIL FAIL github.com/hashicorp/terraform/state/remote 1.496s ok github.com/hashicorp/terraform/terraform 0.971s make: *** [test] Error 1

Is this a common error?

[tomasquith]

@tomasquith - mentioning my other account as I seem to be having issues subscribing from it.

[stack72]

Hi @tasquith @tomasquith

We have gone ahead and merged this. There is no adverse effects here and normal func will continue. When 0.6.16 comes out, please can you test this works as expected and let me know how you get on :)

I am going to try and get an acceptance test in place for this

Paul

[ghost]

Thanks @stack72,

No problem, I'll keep an eye out for this and will test as soon as it lands.

As I can see an image_uri as part of this PR, is that a part of #6494 or am I jumping to the wrong conclusion?

Cheers,

Tom

[ghost]

@stack72 I've grabbed 0.6.16 from the releases service and can confirm that I can provision a machine correctly. The image uri has also allowed me to create a new VHD based on that source as expected in #6494 - I'll test that this works properly today but initial findings look good!

Thanks for all of the help on these!

Tom

[stack72]

@tasquith

This is fantastic news. I am very pleased that this has unblocked you

Paul

[imduffy15]

@tasquith How did you use terraform with charles proxy and get around invalid ssl errors?

[ghost]

@imduffy15

Hi Ian,

All I do is start charles proxy, perform the following steps to trust the certs:

In Charles go to the Help menu and choose "SSL Proxying > Install Charles Root Certificate". Keychain Access will open, and prompt you about the certificate. Click the "Always Trust" button. You will then be prompted for your Administrator password to update the system trust settings.

https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

Then:

You must specifically identify the host names you want to enable SSL Proxying on. The list is in the Proxy Settings, SSL tab. You can also right-click on a host name in the structure view and turn on or off SSL Proxying.

After adding a host name to the SSL Proxying list you may need to restart Charles for existing browser sessions to change.

If you want to SSL Proxy all host names then enter * into the host names list in the SSL Proxying Settings.

https://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Once that's done, you should start to see the api calls to azure, which give you more detailed errors.

Cheers,
Tom

[imduffy15]

Thank you so much!

On Friday 17 June 2016, Tom Asquith [email protected] wrote:

@imduffy15 https://github.com/imduffy15

Hi Ian,

All I do is start charles proxy, perform the following steps to trust the
certs:

In Charles go to the Help menu and choose "SSL Proxying > Install Charles
Root Certificate". Keychain Access will open, and prompt you about the
certificate. Click the "Always Trust" button. You will then be prompted for
your Administrator password to update the system trust settings.

https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

Then:

You must specifically identify the host names you want to enable SSL
Proxying on. The list is in the Proxy Settings, SSL tab. You can also
right-click on a host name in the structure view and turn on or off SSL
Proxying.

After adding a host name to the SSL Proxying list you may need to restart
Charles for existing browser sessions to change.

If you want to SSL Proxy all host names then enter * into the host names
list in the SSL Proxying Settings.

https://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Once that's done, you should start to see the api calls to azure, which
give you more detailed errors.

Cheers,
Tom

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/terraform/issues/6372#issuecomment-226733985,
or mute the thread
https://github.com/notifications/unsubscribe/ABL5K3ZIRzliJT_yauvqbO5tfb4vY84bks5qMnRwgaJpZM4IQ2Rn
.

[stack72]

@imduffy15 / @tasquith

Terraform 0.7 will expose much better error messages from the SDK so don't spend too much time on this ;) it's a temp thing

P.

[imduffy15]

OK for anyone else... on the latest version of OSX this isn't as straight forward as @tasquith described on the latest version of OSX.

The SSL library within golang uses /usr/bin/security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain to discover certificates available to use. On The latest version of OSX you cannot add certificates to the system root keychain.

You must boot your mac into recovery mode, open the terminal from the utilities menu, execute csrutil disable which disables some core osx security stuff.

Then you can reboot and execute sudo security add-trusted-cert -k /System/Library/Keychains/SystemRootCertificates.keychain charles-ssl-proxying-certificate.crt which will add the certificate.

[ghost]

@imduffy15

Cheers for the info! I'm intrigued, I'm using El Capitan which I thought was the latest and didn't have to do any of that!

Either way, all that matters is that you've got it working!

[mcheriyath]

Can someone share the actual code used terraform which worked for using an existing OS disk to create a VM without OS_profile ?

[bbaassssiiee]

You don't need to add to the system root keychain to trust root certificates that are not in there yet. You can ad them to your own (login.keychain) with a flag -r trustRoot

security add-trusted-cert -r trustRoot -p ssl -k "{{ HOME }}/Library/Keychains/login.keychain-db" "myOwnRootCA.cer"