Packer: Azure | azure_arm | custom_data_file for Windows

Created on 20 Jan 2020 · 16Comments · Source: hashicorp/packer

Feature Description

Use of 'custom_data_file' property in the Azure / azure-arm builder on Windows VM's

Use Case(s)

This is creating a blocker for us using the Windows CIS marketplace image which has WinRM disabled by default preventing us using the hardened image.

Example

Builder: "type": "amazon-ebs"

Link: https://blog.petegoo.com/2016/05/10/packer-aws-windows/

Following the steps in this blog post it's possible to workaround these issues on AWS with a Windows AMI, the same steps don't seem to work on Azure

buildeazure enhancement

Source

riweston

👍2

Most helpful comment

Hello,

I had the same kind of issue when I tried to build an Windows image using our already existing provisioning stack. We were using the Azure DSC VM Extension, and I wanted to be able to reuse most of the setup. I created a small Packer provisioner to do that.

I just got notified by Github that https://github.com/hashicorp/packer/issues/6548 was closed, and so I figured out that I should quickly open-source our small provisioner.

Disclaimer: I probably won't maintain it, because we don't use it anymore (it was still used 1 month ago). But hopefully, you can find what you are looking for.

The code is really simple and should work with any Azure VM Extension. It may need to be adapted a little bit for the latest version of Packer (due to HCL2).

Repository: https://github.com/dethi/packer-provisioner-azurerm-vm-extension
An example using DSC Extension: https://github.com/dethi/packer-provisioner-azurerm-vm-extension/blob/master/example/packer_example.json#L56-L62

dethi on 18 Jun 2020

👍2

All 16 comments

Hi! My understanding is that the custom_data_file only pre-installs the script (and doesn't run it) unless cloud-init is installed. I believe cloud-init is not available to you for your use case. So I don't think it is the right workaround in this situation.

Unless Azure has some API tools for starting WinRM on an image, I'm not sure how Packer can fix this for you.

SwampDragons on 22 Jan 2020

Thanks @SwampDragons . As I understand it this is a limitation of Azure VMs, correct?

riweston on 23 Jan 2020

I believe so.

SwampDragons on 23 Jan 2020

akingscote on 8 Feb 2020

This does appear to be an Azure issue but technically you can run a custom script extension on a freshly provisioned server. So if packer had a builder for azure extensions then people could execute via the custom scripts extension. I am sure there are other use cases where people would like to be able to run an azure extension on deployment such as disk encryption.

amarkulis on 5 Jun 2020

Hello @amarkulis I did like to execute a windows script at the time of first boot on a windows machine provisioned using Packer in Azure. Do you happen to know of a workaround for the same?
I just cannot get WinRM communication to work. So i am thinking that there needs to be some configuration done on the Azure VM before Packer can connect to it using WinRM

Thanks,
Kiran Hegde

kirannhegde on 18 Jun 2020

Hello,

I just got notified by Github that https://github.com/hashicorp/packer/issues/6548 was closed, and so I figured out that I should quickly open-source our small provisioner.

Disclaimer: I probably won't maintain it, because we don't use it anymore (it was still used 1 month ago). But hopefully, you can find what you are looking for.

The code is really simple and should work with any Azure VM Extension. It may need to be adapted a little bit for the latest version of Packer (due to HCL2).

dethi on 18 Jun 2020

👍2

Hello,

I had the same kind of issue when I tried to build an Windows image using our already existing provisioning stack. We were using the Azure DSC VM Extension, and I wanted to be able to reuse most of the setup. I created a small Packer provisioner to do that.

I just got notified by Github that #6548 was closed, and so I figured out that I should quickly open-source our small provisioner.

Disclaimer: I probably won't maintain it, because we don't use it anymore (it was still used 1 month ago). But hopefully, you can find what you are looking for.

The code is really simple and should work with any Azure VM Extension. It may need to be adapted a little bit for the latest version of Packer (due to HCL2).

Repository: https://github.com/dethi/packer-provisioner-azurerm-vm-extension
An example using DSC Extension: https://github.com/dethi/packer-provisioner-azurerm-vm-extension/blob/master/example/packer_example.json#L56-L62

Hello I've implemented your solution (and updated the code to support the new packer version, if it works ill make a PR) but still getting:

=> azure-arm: Validating deployment template ... 2020-07-29T08:59:53.2282094Z 2020-07-29T08:59:53.2305303Z ==> azure-arm: -> ResourceGroupName : 'pkr-Resource-Group-ydy74quqop' 2020-07-29T08:59:53.2305583Z 2020-07-29T08:59:53.2308005Z ==> azure-arm: -> DeploymentName : 'pkrdpydy74quqop' 2020-07-29T08:59:53.2308171Z 2020-07-29T08:59:57.8270891Z ==> azure-arm: Deploying deployment template ... 2020-07-29T08:59:57.8271088Z 2020-07-29T08:59:57.8280882Z ==> azure-arm: -> ResourceGroupName : 'pkr-Resource-Group-ydy74quqop' 2020-07-29T08:59:57.8281910Z ==> azure-arm: -> DeploymentName : 'pkrdpydy74quqop' 2020-07-29T08:59:57.8282028Z 2020-07-29T09:03:01.4885907Z ==> azure-arm: Getting the VM's IP address ... 2020-07-29T09:03:01.4886218Z 2020-07-29T09:03:01.4887396Z ==> azure-arm: -> ResourceGroupName : 'pkr-Resource-Group-ydy74quqop' 2020-07-29T09:03:01.4887829Z ==> azure-arm: -> PublicIPAddressName : 'pkripydy74quqop' 2020-07-29T09:03:01.4888255Z ==> azure-arm: -> NicName : 'pkrniydy74quqop' 2020-07-29T09:03:01.4888747Z 2020-07-29T09:03:01.4893599Z ==> azure-arm: -> Network Connection : 'PublicEndpoint' 2020-07-29T09:03:01.4893800Z 2020-07-29T09:03:01.5949184Z ==> azure-arm: -> IP Address : '52.143.130.218' 2020-07-29T09:03:01.5949325Z 2020-07-29T09:03:01.5966568Z ==> azure-arm: Waiting for WinRM to become available... 2020-07-29T09:03:01.5966714Z 2020-07-29T09:06:01.6207700Z ==> azure-arm: Timeout waiting for WinRM. 2020-07-29T09:06:01.6210118Z ==> azure-arm: Removing the created Deployment object: 'pkrdpydy74quqop' 2020-07-29T09:06:01.6210587Z 2020-07-29T09:06:17.1254497Z ==> azure-arm: Removing the created Deployment object: 'kvpkrdpydy74quqop' 2020-07-29T09:06:17.1254938Z 2020-07-29T09:06:32.3790635Z ==> azure-arm: 2020-07-29T09:06:32.3792388Z 2020-07-29T09:06:32.3793715Z ==> azure-arm: Cleanup requested, deleting resource group ... 2020-07-29T09:06:32.3793814Z 2020-07-29T09:13:18.3563956Z ==> azure-arm: Resource group has been deleted. 2020-07-29T09:13:18.3564358Z Build 'azure-arm' errored: Timeout waiting for WinRM. 2020-07-29T09:13:18.3564440Z 2020-07-29T09:13:18.3564480Z ==> Some builds didn't complete successfully and had errors: 2020-07-29T09:13:18.3564523Z --> azure-arm: Timeout waiting for WinRM. 2020-07-29T09:13:18.3564547Z 2020-07-29T09:13:18.3564614Z ==> Builds finished but no artifacts were created.

If i'm not mistaken the provisioner part still requires WINRM to communicate to the VM right? if WinRM is deactivated by default it is impossible to connect to the machine, in my case I'm using a CIS image and it seems I can't use packer.

Any help would be greatly appreciated.

FreddyAyala on 29 Jul 2020

Hello @dethi Sincere apologies for the delayed response. Thanks a ton for open sourcing your code. Much appreciated.

kirannhegde on 29 Jul 2020

❤1

I've confirmed that when packer is waiting for WinRM to become available if I go to azure portal, search the deployed VM and then click the option => Run Command => Run powershell command:

write-output "Running User Data Script"
write-host "(host) Running User Data Script"

Set-ExecutionPolicy Unrestricted -Scope LocalMachine -Force -ErrorAction Ignore

# Don't set this before Set-ExecutionPolicy as it throws an error
$ErrorActionPreference = "stop"

# Remove HTTP listener
Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse

# Create a self-signed certificate to let ssl work
$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "packer"
New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force

# WinRM
write-output "Setting up WinRM"
write-host "(host) setting up WinRM"

cmd.exe /c winrm quickconfig -q
cmd.exe /c winrm set "winrm/config" '@{MaxTimeoutms="1800000"}'
cmd.exe /c winrm set "winrm/config/winrs" '@{MaxMemoryPerShellMB="1024"}'
cmd.exe /c winrm set "winrm/config/service" '@{AllowUnencrypted="true"}'
cmd.exe /c winrm set "winrm/config/client" '@{AllowUnencrypted="true"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/client/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{CredSSP="true"}'
cmd.exe /c winrm set "winrm/config/listener?Address=*+Transport=HTTPS" "@{Port=`"5986`";Hostname=`"packer`";CertificateThumbprint=`"$($Cert.Thumbprint)`"}"
cmd.exe /c netsh advfirewall firewall set rule group="remote administration" new enable=yes
cmd.exe /c netsh firewall add portopening TCP 5986 "Port 5986"
cmd.exe /c net stop winrm
cmd.exe /c sc config winrm start= auto
cmd.exe /c net start winrm

#End WinRM

Then the process is finished successfully. With the az cli we would use the command:

az vm run-command invoke  --command-id RunPowerShellScript --name win-vm -g my-resource-group  \
    --scripts 'param([string]$arg1,[string]$arg2)' \
    'Write-Host This is a sample script with parameters $arg1 and $arg2' \
    --parameters 'arg1=somefoo' 'arg2=somebar'

Is there any way to integrate this in the packer build phase?

FreddyAyala on 30 Jul 2020

@FreddyAyala I hit exactly the same issue today, using packer to build a Windows 2019 server in Azure. Did you find a solution?

david-wells-1 on 3 Dec 2020

@FreddyAyala I hit exactly the same issue today, using packer to build a Windows 2019 server in Azure. Did you find a solution?

Hello from our side the only solution was to create an Automation Account that will detect when a virtual machine is deployed using a tag packer=true and then it will execute a powershell script that enables WinRM. No other solution has worked and we tried very hard.

FreddyAyala on 4 Dec 2020

Hello, @david-wells-1 @FreddyAyala Is this an issue with WinRM being enabled? Could you please state your exact scenario? When I initially commented on this thread, I had issues getting Packer to run with Winrm authentication. However, I have resolved that problem quite some time back. So I would be interested to know about the issue that you are facing so that I can share my learning. I don't know if you are using a hardened image where WinRM is disabled and hence you have issues. I am using the images provided by Microsoft.

kirannhegde on 4 Dec 2020

Hi @kirannhegde We are using Packer to build azure Windows 2019 VM and WinRM connection always timeout. Using an existing Vault so just looking at creating a self-signed cert as this may be missing, preventing the WinRM connection.

david-wells-1 on 4 Dec 2020

Here is what i have in my Packer json file:

"communicator": "winrm",
"winrm_use_ssl": true,
"winrm_insecure": true,
"winrm_timeout": "30m",
"winrm_username": "{{user `winrm_username`}}",

Is yours the same and yet you are facing this issue?
In my case, I allow Packer to create the key vault and use secrets from the Packer created key vault.
Is there a reason you don't want to do that?

Regards,
Kiran Hegde

kirannhegde on 4 Dec 2020

From what I can tell, Packer itself configures WinRM on the VM when using the azure-arm builder, and there shouldn't any need to preconfigure WinRM when using the azure-arm builder. You can see this if you extract the ARM template for the VM from the portal:

                "osProfile": {
                    "computerName": "[parameters('virtualMachines_pkrvm2av02x8tt7_name')]",
                    "adminUsername": "packer",
                    "windowsConfiguration": {
                        "provisionVMAgent": true,
                        "enableAutomaticUpdates": true,
                        "winRM": {
                            "listeners": [
                                {
                                    "protocol": "https",
                                    "certificateUrl": "https://pkrkv2av02x8tt7.vault.azure.net/secrets/packerKeyVaultSecret/b933844dc9eb4114ad0aebd376f85720"
                                }
                            ]
                        }
                    },
                    "secrets": [
                        {
                            "sourceVault": {
                                "id": "[parameters('vaults_pkrkv2av02x8tt7_externalid')]"
                            },
                            "vaultCertificates": [
                                {
                                    "certificateUrl": "https://pkrkv2av02x8tt7.vault.azure.net/secrets/packerKeyVaultSecret/b933844dc9eb4114ad0aebd376f85720",
                                    "certificateStore": "My"
                                }
                            ]
                        }
                    ]
                },
                "networkProfile": {
                    "networkInterfaces": [
                        {
                            "id": "[parameters('networkInterfaces_pkrni2av02x8tt7_externalid')]"
                        }
                    ]
                },
                "diagnosticsProfile": {
                    "bootDiagnostics": {
                        "enabled": false
                    }
                }
            }

Essentially what it's doing under the hood is performing these steps: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/winrm. It would be useful if Packer included this detail in their documentation for the azure-arm builder, as it's not obvious and caused us quite a bit of confusion.

The configuration options for WinRM using ARM are quite limited: https://docs.microsoft.com/en-us/azure/templates/microsoft.compute/virtualmachines#winrmconfiguration-object

Regarding custom data, indeed it looks as if this is not currently possible using Windows VMs: https://docs.microsoft.com/en-us/azure/virtual-machines/custom-data#windows