Packer: scp error provisioning Powershell script for Windows guest

Created on 2 Mar 2018  ·  52Comments  ·  Source: hashicorp/packer

When using packer on a Linux host with a Windows 10 (or Win2016 and Win7) guest (which has the Cygwin SSH server installed) using the qemu builder and powershell "scripts" provisioner, packer cannot seem to successfully copy the script (or scripts, but in this case singular) to the guest:

==> qemu: Waiting for SSH to become available...
==> qemu: Connected to SSH!
==> qemu: Provisioning with Powershell...
==> qemu: Provisioning with powershell script: /mnt/LinStorage/VMs/packertempPacker-Windows10-kvm/CustomScripts/Win-provision.ps1
==> qemu: Deleting output directory...
Build 'qemu' errored: Error processing command: Error uploading ps script containing env vars: scp: ./${env:SYSTEMROOT}\Temp\packer-env-vars-5a999ccb-43f2-9f0b-2ebd-5265f410a14a.ps1: No such file or directory

This functionality worked (and continues to work) in v1.1.3, however has not worked in v1.2.0 or v1.2.1. I am assuming this is not a problem with my SSH server in the guest (since v1.1.3 was successful in uploading scripts), however I noted the change to the provisioning using files (GH-5426), and don't know if that affects me or if I need to change something in my json file to make it work properly. Cygwin's SSH server does not use standard Windows file pathing or environment variables, but then again I don't think packer explicitly knows the guest is Windows.

Details:

provisionepowershell waiting-reply windows-guest

Most helpful comment

I've created a new issue, https://github.com/hashicorp/packer/issues/5994, to track the test -d thing.

All 52 comments

The powershell provisioner contains explicit assumptions that it's running on a windows guest, and assumes windows pathing in several places. Looking over this, I'm wondering if it's as simple as having slashes going the wrong direction. With cygwin pathing, is %SYSTEMROOT% set? If yes, then maybe this attached binary
packer.tar.gz
will work?

if you would rather build yourself, the relevant commit is here. https://github.com/hashicorp/packer/pull/5973

I've had the same issue (running packer on linux and provisioning an OpenStack image on win 2k16 running an SSH server) but the build you provided made no difference. When it says: ./${env:SYSTEMROOT}\Temp\packer-env-vars-5a999ccb-43f2-9f0b-2ebd-5265f410a14a.ps1: No such file or directory, is that the file on the guest or the host? I've looked on the guest machine after I got this failure, and while there is packer-stuff in ${env:SYSTEMROOT}\Temp, there is no temp file like that.

Yeah, that file is on the guest. And I just noticed in your error message there it's using relative pathing, which is definitely wrong. @lbergnehr are you also using cygwin?

@lbergnehr are you also using cygwin?

Yup.

@DanHam I don't suppose you have a cygwin setup lying around that would make this ticket something you can handle more easily than I can? Calling on you because this is caused by https://github.com/hashicorp/packer/pull/5872; if you don't have time I'll work on it some more but lacking a cygwin setup means it'll be slow for me.

@SwampDragons I have tested the build you have attached, and it seems to be working now. As a side note, %SYSTEMROOT% (or rather, ${env:SYSTEMROOT}) does not echo any value in the Cygwin bash terminal as an environment variable. As a reference, here is the debug output from packer:

packer: 2018/03/06 17:25:01 Uploading env vars to ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1
packer: 2018/03/06 17:25:01 [INFO] 103 bytes written for 'uploadData'
packer: 2018/03/06 17:25:01 opening new ssh session
[INFO] 103 bytes written for 'uploadData'
packer: 2018/03/06 17:25:01 starting remote command: test -d "${env:SYSTEMROOT}/Temp/packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1"
packer: 2018/03/06 17:25:01 Remote command exited with '1': test -d "${env:SYSTEMROOT}/Temp/packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1"
packer: 2018/03/06 17:25:01 opening new ssh session
packer: 2018/03/06 17:25:01 Starting remote scp process:  scp -vt ${env:SYSTEMROOT}/Temp
packer: 2018/03/06 17:25:01 Started SCP session, beginning transfers...
packer: 2018/03/06 17:25:01 Copying input data into temporary file so we can read the length
packer: 2018/03/06 17:25:01 [DEBUG] scp: Uploading packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1: perms=C0644 size=103
packer: 2018/03/06 17:25:01 SCP session complete, closing stdin pipe.
packer: 2018/03/06 17:25:01 Waiting for SSH session to complete.
packer: 2018/03/06 17:25:01 scp stderr (length 73): Sink: C0644 103 packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1
packer: 2018/03/06 17:25:01 [INFO] 17063 bytes written for 'uploadData'
packer: 2018/03/06 17:25:01 opening new ssh session
[INFO] 17063 bytes written for 'uploadData'
packer: 2018/03/06 17:25:01 starting remote command: test -d "c:/Windows/Temp/script-5a9f128a-7da1-8518-2de7-b7c1054ebb07.ps1"
packer: 2018/03/06 17:25:02 Remote command exited with '1': test -d "c:/Windows/Temp/script-5a9f128a-7da1-8518-2de7-b7c1054ebb07.ps1"
packer: 2018/03/06 17:25:02 opening new ssh session
packer: 2018/03/06 17:25:02 Starting remote scp process:  scp -vt c:/Windows/Temp
packer: 2018/03/06 17:25:02 Started SCP session, beginning transfers...
packer: 2018/03/06 17:25:02 Copying input data into temporary file so we can read the length
packer: 2018/03/06 17:25:02 [DEBUG] scp: Uploading script-5a9f128a-7da1-8518-2de7-b7c1054ebb07.ps1: perms=C0644 size=17063
packer: 2018/03/06 17:25:02 SCP session complete, closing stdin pipe.
packer: 2018/03/06 17:25:02 Waiting for SSH session to complete.
packer: 2018/03/06 17:25:02 scp stderr (length 66): Sink: C0644 17063 script-5a9f128a-7da1-8518-2de7-b7c1054ebb07.ps1
packer: 2018/03/06 17:25:02 opening new ssh session
packer: 2018/03/06 17:25:02 starting remote command: powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9f153d-46c2-8075-a833-c2382ab88ccd.ps1; &'c:/Windows/Temp/script-5a9f128a-7da1-8518-2de7-b7c1054ebb07.ps1';exit $LastExitCode }"

Thanks very much, I look forward to this fix in the next packer release.

Oh! Interesting that it worked for you. Okay, @lbergnehr can you provide your full debug log, since my build seems to work for the original issue creator? Are you setting the ssh communicator in your Packer config?

It looks like my error might be slightly different, or at least I'm running as elevated user. Here's the debug log:

2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Uploading env vars to ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1 2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 [INFO] 64 bytes written for 'uploadData'
2018/03/07 10:10:56 [INFO] 64 bytes written for 'uploadData'
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 opening new ssh session
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 starting remote command: test -d "${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1"
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Remote command exited with '1': test -d "${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1"
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 opening new ssh session
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Starting remote scp process:  scp -vt ${env:SYSTEMROOT}/Temp
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Started SCP session, beginning transfers...
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Copying input data into temporary file so we can read the length
2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 [DEBUG] scp: Uploading packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1: perms=C0644 size=64
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 SCP session complete, closing stdin pipe.
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Waiting for SSH session to complete.
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 scp stderr (length 72): Sink: C0644 64 packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Building elevated command wrapper for: powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1; &'c:/Windows/Temp/script-5a9fb9dd-d30a-9733-8f53-689ab44bcc62.ps1'; exit $LastExitCode }"
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Command [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1; &'c:/Windows/Temp/script-5a9fb9dd-d30a-9733-8f53-689ab44bcc62.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fbab1-22d6-8775-4e88-79e959b5611d.out 2>&1] converted to [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1; &'c:/Windows/Temp/script-5a9fb9dd-d30a-9733-8f53-689ab44bcc62.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fbab1-22d6-8775-4e88-79e959b5611d.out 2>&1] for use in XML string
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Uploading elevated shell wrapper for command [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1; &'c:/Windows/Temp/script-5a9fb9dd-d30a-9733-8f53-689ab44bcc62.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fbab1-22d6-8775-4e88-79e959b5611d.out 2>&1] to [${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1]
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 [INFO] 2773 bytes written for 'uploadData'
2018/03/07 10:10:57 [INFO] 2773 bytes written for 'uploadData'
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 opening new ssh session
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 starting remote command: test -d "${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1"
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Remote command exited with '1': test -d "${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1"
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 opening new ssh session
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Starting remote scp process:  scp -vt .
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Started SCP session, beginning transfers...
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Copying input data into temporary file so we can read the length
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 [DEBUG] scp: Uploading ${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1: perms=C0644 size=2773
2018/03/07 10:10:57 [INFO] (telemetry) ending powershell
2018/03/07 10:10:57 [INFO] (telemetry) ending null
2018/03/07 10:10:57 ui error: Build 'null' errored: Error processing command: Error generating elevated runner: Error preparing elevated powershell script: scp: ./${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1: No such file or directory
Build 'null' errored: Error processing command: Error generating elevated runner: Error preparing elevated powershell script: scp: ./${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1: No such file or directory

Also, this might be interesting after ssh:ing to my guest:

$ echo $SYSTEMROOT
C:\Windows
$ cmd /C echo %SYSTEMROOT%
C:\Windows

Also, as I mentioned previously, the file cannot be found in the temp dir afterwards:

$ ls -l C:/Windows/Temp | findstr packer
drwxr-xr-x 1 Administrator None         0 Mar  5 14:37 packer-chef-solo

@lbergnehr @SwampDragons Hello! I wonder if lbergnehr is running into the same issue described HERE and HERE. This would not be related to the original problem of 'slashes in the wrong direction' that @ramesh45345 was having - Sorry about the issues caused there.

I noticed in lbergnehr's debug output that ${env:TEMP} is being used in the path for uploading the script. This path will evaluate differently for the Windows user used to upload the script e.g. C:\Users\packer\AppData\Local\Temp and the elevated user that tries to run it e.g. C:\Users\ADMINI~1\AppData\Local\Temp as it is a 'User Context' rather than 'System Context' environment variable. The difference causes the No such file or directory error.

@lbergnehr - Could you look under C:\Users[YourUser]\AppData\Local\Temp for your script? Also did you set the upload path for scp or was that set by Packer for you?

Unfortunately I don't have an environment available in which I can replicate this... but that's my best guess for now.

See THIS technet article for a much better explanation of User and System context environment variables.

Hi @DanHam !

There are no packer files in any use temp directory:

PS> ls -recurse -filter *packer* /Users
(nothing)

I actually tried running packer in Windows as well, which produced this other error log:

2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Uploading env vars to ${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 [INFO] 64 bytes written for 'uploadData'
2018/03/07 13:12:28 [INFO] 64 bytes written for 'uploadData'
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 opening new ssh session
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 starting remote command: test -d "${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1"
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Remote command exited with '1': test -d "${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1"
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 opening new ssh session
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Starting remote scp process:  scp -vt ${env:SYSTEMROOT}/Temp
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Started SCP session, beginning transfers...
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Copying input data into temporary file so we can read the length
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 [DEBUG] scp: Uploading packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1: perms=C0644 size=64
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 SCP session complete, closing stdin pipe.
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Waiting for SSH session to complete.
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 scp stderr (length 72): Sink: C0644 64 packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Building elevated command wrapper for: powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1; &'c:/Windows/Temp/script-5a9fd6eb-0fb9-fcdb-3728-e4452ebe324a.ps1'; exit $LastExitCode }"
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Command [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1; &'c:/Windows/Temp/script-5a9fd6eb-0fb9-fcdb-3728-e4452ebe324a.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fd72c-fad4-b472-11a6-a176807288e2.out 2>&1] converted to [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1; &'c:/Windows/Temp/script-5a9fd6eb-0fb9-fcdb-3728-e4452ebe324a.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fd72c-fad4-b472-11a6-a176807288e2.out 2>&1] for use in XML string
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Uploading elevated shell wrapper for command [powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. ${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fd72c-723f-a8e5-69fa-240b9029cf5f.ps1; &'c:/Windows/Temp/script-5a9fd6eb-0fb9-fcdb-3728-e4452ebe324a.ps1'; exit $LastExitCode }" > %SYSTEMROOT%\Temp\packer-5a9fd72c-fad4-b472-11a6-a176807288e2.out 2>&1] to [${env:TEMP}\packer-elevated-shell-5a9fd72c-4608-a226-a971-aac5aaf256ea.ps1]
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 [INFO] 2773 bytes written for 'uploadData'
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 opening new ssh session
2018/03/07 13:12:28 [INFO] 2773 bytes written for 'uploadData'
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 starting remote command: test -d "${env:TEMP}\packer-elevated-shell-5a9fd72c-4608-a226-a971-aac5aaf256ea.ps1"
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Remote command exited with '1': test -d "${env:TEMP}\packer-elevated-shell-5a9fd72c-4608-a226-a971-aac5aaf256ea.ps1"
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 opening new ssh session
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Starting remote scp process:  scp -vt ${env:TEMP}
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Started SCP session, beginning transfers...
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 Copying input data into temporary file so we can read the length
2018/03/07 13:12:28 packer.exe: 2018/03/07 13:12:28 [DEBUG] scp: Uploading packer-elevated-shell-5a9fd72c-4608-a226-a971-aac5aaf256ea.ps1: perms=C0644 size=2773
2018/03/07 13:12:28 [INFO] (telemetry) ending powershell
2018/03/07 13:12:28 [INFO] (telemetry) found error: Error processing command: Error generating elevated runner: Error preparing elevated powershell scBuild 'null' errored: Error processing command: Error generating elevated runner: Error preparing elevated powershell script: scp: ambiguous target
ript: scp: ambiguous target

The ambiguous target error is described here as being caused by there being spaces in the scp target, which is kind of strange to me...

Moreover, I'm not setting any paths myself but letting packer handle it. Here's the provisioning part of my template:

    {
      "type": "powershell",
      "script": "windows_updates.ps1",
      "elevated_user": "Administrator",
      "elevated_password": "[REDACTED]"
    }

Also, note that the chef-solo provisioner I'm running before this provisioner is succeeding scp:ing stuff onto the guest (from reading the logs at least).

Note that the above test on Windows used version 1.2.0, not the patch you provided, @SwampDragons .

Another thing I noticed is from my packer on linux log it's running test -d "${env:TEMP}\packer-elevated-shell-5a9fd72c-4608-a226-a971-aac5aaf256ea.ps1" (on a file) while the man page for test -d says:

-d FILE
FILE exists and is a directory

but I'm just reading the logs here so maybe that's expected. @ramesh45345 even have that in their log and the command apparently exits with error code 1 but it allegedly works there.

Even stranger, it looks like test -d is running even though I'm running a Windows guest? Does it know I'm running cygwin, because otherwise I would think test doesn't exist on Windows.

Sorry for spamming. Really thankful you're looking into this!

@lbergnehr Hmm... the plot thickens! Would you be able to provide the full template you are using (without any passwords of course!) and the full debug logs?

I'm not promising anything as I'm rather busy with other stuff at the moment but I would like to try and get to the bottom of this. If I can reproduce your environment (and the errors) that would be a great help.

This same issue has been driving me crazy for the last 3 days:

Build 'virtualbox-iso' errored: Error processing command: Error uploading ps script containing env vars: scp: ./${env:SYSTEMROOT}\Temp\packer-env-vars-5a9fff06-4fc0-3b96-522b-cc107cc441b0.ps1: No such file or directory

I looked at #5872. I am running virtualbox on OSX. I used homebrew to install packer could it be that homebrew had not updated the merged commit.

packer 1.2.1

Removed homebrew/packer and downloaded from site. same issue....

oh, the test -d thing is interesting, and it means cygwin vs not-cygwin is probably a red herring. It's happening inside the SSH communicator before the upload, to make sure that the files are present. This must have assumed incorrectly that anyone using the SSH communicator would be on a linux OS. Let me see what I can do to mitigate this. Will upload a new binary for testing soon.

downgraded to packer 1.1.3. Was able to get past error but now I get.

virtualbox-iso: C:\Users\vagrant>powershell -Command "(New-Object System.Net.WebClient).DownloadFile('https://github.com/downloads/alexkasko/openjdk-unofficial-builds/openjdk-1.6.0-unofficial-b27-windows-amd64.zip', 'C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip')" 0 virtualbox-iso: Exception calling "DownloadFile" with "2" argument(s): "The request was
virtualbox-iso: aborted: Could not create SSL/TLS secure channel."
virtualbox-iso: At line:1 char:1
virtualbox-iso: + (New-Object System.Net.WebClient).DownloadFile('https://github.com/do ...
virtualbox-iso: + ~~~~~~~~~~~~~~~~~
virtualbox-iso: + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
virtualbox-iso: + FullyQualifiedErrorId : WebException
virtualbox-iso:
virtualbox-iso:
virtualbox-iso: C:\Users\vagrant>cmd /c ""C:\Program Files\7-Zip\7z.exe" x "C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip" -oC:\openjdk6"
virtualbox-iso:
virtualbox-iso: ERROR: The system cannot find the file specified.
virtualbox-iso: C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip
virtualbox-iso:
virtualbox-iso: 7-Zip 18.01 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-01-28
virtualbox-iso:
virtualbox-iso: Scanning the drive for archives:
virtualbox-iso:
virtualbox-iso:
virtualbox-iso:
virtualbox-iso: System ERROR:
virtualbox-iso: The system cannot find the file specified.
==> virtualbox-iso: Deregistering and deleting VM...
==> virtualbox-iso: Deleting output directory...
Build 'virtualbox-iso' errored: Script exited with non-zero exit status: 2

==> Some builds didn't complete successfully and had errors:
--> virtualbox-iso: Script exited with non-zero exit status: 2

==> Builds finished but no artifacts were created.

@em0ral Did you manage to test with the binary that Swampdragons prepared for ramesh45345 to solve this issue? It looks as though you were suffering with the exact problem in the 1.2.0 version of Packer.

See the comment above HERE for the link to the binary (packer.tar.gz)

Let us know if that helps.

Says cannot open binary file.

em0ral:> ll packer
-rwxr-xr-x@ 1 em0ral staff 85M Mar 5 16:26 packer

em0ral:> ll ~/packer
-rwxr-xr-x@ 1 em0ral staff 87M Feb 23 11:32 /Users/em0ral/packer

The second binary was downloaded from the website...excuse my noobness

Dammit - Sorry, that binary is for a Linux based system...

I've uploaded a binary for OSX [EDIT: Now deleted] that has the fix from Swampdragons. NOTE: I am NOT an employee of Hashicorp - just a contributor to Packer... so only download if you trust me :)

You will need to run:

chmod +x packer 

Then try

./packer version

The output should be Packer v1.2.2-dev (75c1133c1)

Thank you...testing in my OSX VM...will let you know shortly.

Fixed the original error. But I ended up with the same error I had when I downgraded to 1.1.3. Having too much trouble with this metasploit3 installation

virtualbox-iso: Provisioning with shell script: scripts/installs/install_openjdk6.bat
virtualbox-iso:
virtualbox-iso: C:\Users\vagrant>powershell -Command "(New-Object System.Net.WebClient).DownloadFile('https://github.com/downloads/alexkasko/openjdk-unofficial-builds/openjdk-1.6.0-unofficial-b27-windows-amd64.zip', 'C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip')" 0 virtualbox-iso: Exception calling "DownloadFile" with "2" argument(s): "The request was
virtualbox-iso:
virtualbox-iso: aborted: Could not create SSL/TLS secure channel."
virtualbox-iso: At line:1 char:1
virtualbox-iso: + (New-Object System.Net.WebClient).DownloadFile('https://github.com/do ...
virtualbox-iso: C:\Users\vagrant>cmd /c ""C:\Program Files\7-Zip\7z.exe" x "C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip" -oC:\openjdk6"
virtualbox-iso: + ~~~~~~~~~~~~~~~~~
virtualbox-iso: + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
virtualbox-iso: + FullyQualifiedErrorId : WebException
virtualbox-iso:
virtualbox-iso:
virtualbox-iso: ERROR: The system cannot find the file specified.
virtualbox-iso:
virtualbox-iso: 7-Zip 18.01 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-01-28
virtualbox-iso: C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip
virtualbox-iso:
virtualbox-iso:
virtualbox-iso:
virtualbox-iso:
virtualbox-iso: System ERROR:
virtualbox-iso: The system cannot find the file specified.
virtualbox-iso: Scanning the drive for archives:
==> virtualbox-iso: Deregistering and deleting VM...
==> virtualbox-iso: Deleting output directory...
Build 'virtualbox-iso' errored: Script exited with non-zero exit status: 2

==> Some builds didn't complete successfully and had errors:
--> virtualbox-iso: Script exited with non-zero exit status: 2

==> Builds finished but no artifacts were created.
Error building the Vagrant box using Packer. Please check the output above for any error messages.

So in that case it's a download failing. @em0ral are you using the ssh or the winrm communicator?

ssh

@em0ral Well.... so my reading of that is that the problem now seems to be fixed from a Packer perspective and the problem now lies with your script? In other words Swampdragons fix worked for you?

oh, Dan's right. It looks like the issue is now the download being called by the script, not the provisioner. Is 'C:\Windows\Temp\openjdk-1.6.0-unofficial-b27-windows-amd64.zip' present? Should it have been uploaded by Packer previously?

Of that I am not sure. I will post this error for the metasploit3 maintainers....I thank you guys.

Okay, sounds good. I am going to merge my PR then, and the patch fixing the original problem will be released with 1.2.2. I'm also going to close this ticket, since the problem stated in the ticket description has been fixed. If you run into more issues that look like Packer's fault, open up another ticket with a full repro case. Thanks all!

@SwampDragons There's still lbergnehr's issue... although I'm not sure that's related to the original ticket...

I've also tested your fix (just a basic run through with some of my Windows build) and all looks good to me!

Ack good point I lost him in the conversation. @lbergnehr I have a new binary for you here -- Can you test it out and let me know the results?
packer.tar.gz

EDIT: Ignore the rest of this unless you're bored 😄 . I think I've found the root cause of the issue - the backslashes in the powershell path and the changes in #5782 were a bit of a red herring. I think #5426 may have introduced the problem... see last comment below.
EDIT: I take that back... #5426 may not be to blame... this could be a bug in Go??

@SwampDragons Been taking a deeper look at this. I noticed from lbergnehr's first bit of debug output that the upload path seems to be getting messed up by the communicator e.g. we've gone from an upload path of ${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1 within the powershell provisioner to . within the scpUploadSession func in the ssh communicator. The scp -vt . should instead be scp -vt ${env:TEMP}

2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Uploading elevated shell wrapper for command BLAH...BLAH to [${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1]
...
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 opening new ssh session
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 starting remote command: test -d "${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1"
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Remote command exited with '1': test -d "${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1"
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 opening new ssh session
2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Starting remote scp process:  scp -vt .

I also noticed that the scpUploadSession function has code to fix paths with backward slashes HERE.

Long and short, I think the backslashes in the path within the powershell provisioner may have been a red herring as those backslashes _should_ have been fixed and converted by that code. I'm also fairly sure we were already uploading and 'dot sourcing' env vars for the elevated powershell command before #5782 - so that path with backslashes would have already been in use before #5872 was merged. #5872 simply extended this functionality to the standard (non-elevated) powershell command.

I'm wondering if the errors being experienced by @ramesh45345 and @lbergnehr are instead related to #5426? Perhaps the changes in that PR are somehow preventing the 'backslash fix' from being applied? As noted above, there is definitely something weird happening to the upload path within that function...

I'm wondering if we should hold off on the merge of your PR until we've investigated the behaviour of the scpUploadSession function a bit further and also try and see how that test -d affects things...

EDIT: Noticed that the scp upload seems to proceed fine for uploading the env vars - note this path has forward slashes:

2018/03/07 10:10:56 packer: 2018/03/07 10:10:56 Uploading env vars to ${env:SYSTEMROOT}/Temp/packer-env-vars-5a9fbab0-debf-e34c-2e68-c7f6202eda4c.ps1

but fails for the elevated shell script that uses back slashes...

2018/03/07 10:10:57 packer: 2018/03/07 10:10:57 Uploading elevated shell wrapper for command [BLAH...BLAH] to [${env:TEMP}\packer-elevated-shell-5a9fbab1-5d8a-2ba3-b291-6db01dc235b7.ps1]

Stating the obvious I guess, but it seems like what ever has changed in the communicator code works with paths that use forward slashes but fails if the path contains backslashes.

@SwampDragons ...of course you could be way ahead of me already... 😃

Sorry for spamming (AND all the edit spam), but I think I got it... maybe... I hope...

  • filepath.Dir(path) returns a . when the path uses backslashes.
  • filepath.Base(path) simply returns the whole path back when the path uses backslashes.

Anyhow, in the scpUploadSession function if we supply a path with backslashes e.g. ${env:SYSTEMROOT}\Temp\packer-env-vars.sh then unfortunately...

func (c *comm) scpUploadSession(path string, input io.Reader, fi *os.FileInfo) error {

    // The target directory and file for talking the SCP protocol
    target_dir := filepath.Dir(path)
    target_file := filepath.Base(path)

target_dir gets set to . which clearly is a bit of an problem for us!
target_file gets set to ${env:SYSTEMROOT}\Temp\packer-env-vars.sh which again is a bit of a problem!

I think this is probably all happening because the filepath package makes use of os.PathSeparator. For instance filepath.ToSlash won't (always) work for us as it eventually uses os.PathSeparator to determine what separator to replace in the path. See HERE and HERE

A fix could (perhaps) be:

func (c *comm) scpUploadSession(path string, input io.Reader, fi *os.FileInfo) error {

    // Convert backslashes in path to forward slashes [Some better comment here]
    path = strings.Replace(path, `\`, `/`, -1)

    // The target directory and file for talking the SCP protocol
    target_dir := filepath.Dir(path)
    target_file := filepath.Base(path)

Although I can see a *nix style path with an escaped space causing issues...

May also be worth adding something to the tests to check scpUploadSession correctly handles backslash escaped paths in future...

Incidentally, the test -d command seems to be working fine. test -d returns 0 if the path is a directory and 1 otherwise...

@DanHam awesome deep dive, thanks. I'll take a closer look at the ssh communicator when I log on tomorrow.

Hi - sorry to add to this but I'm trying with packer 1.2.0 (and also 1.2.1) (on windows 10) to upload files to a windows docker container built by packer. I'm also suffering from an upload problem very similar to this:

Shall I open a new issue or is it similar enough?

output...
C:\Users\simon\packer\packer_templates\hypv-and-docker-win2012r2>packer build -debug docker-wsc-volume.json
Debug mode enabled. Builds will not be parallelized.
dockerwsc output will be in this color.
==> dockerwsc: Creating a temporary directory for sharing data...
==> dockerwsc: Pausing after run of step 'StepTempDir'. Press enter to continue.
==> dockerwsc: Pulling Docker image: microsoft/windowsservercore
dockerwsc: Using default tag: latest
dockerwsc: latest: Pulling from microsoft/windowsservercore
dockerwsc: Digest: sha256:554b69722f31381f10a2f3ddd81e4cb50beba7af8561e9014a9e431a52c8f825
dockerwsc: Status: Image is up to date for microsoft/windowsservercore:latest
==> dockerwsc: Pausing after run of step 'StepPull'. Press enter to continue.
==> dockerwsc: Starting docker container...
dockerwsc: Run command: docker run -v c:/Users/simon/packer/packer_templates/hypv-and-docker-win2012r2/windows/common:c:/common -v c:/tmp/packer-docker351160971:c:/packerfiles -dit microsoft/windowsservercore powershell
dockerwsc: Container ID: c962a66a011043520e4faa24ddeb2f6bd5ba22d3937f763a5063585225aa4143
==> dockerwsc: Pausing after run of step 'StepRun'. Press enter to continue.
==> dockerwsc: Pausing after run of step 'StepConnect'. Press enter to continue.
==> dockerwsc: Provisioning with Powershell...
==> dockerwsc: Provisioning with powershell script: C:\Users\simon\AppData\Local\Temp\packer-powershell-provisioner028567271
==> dockerwsc: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
==> dockerwsc: Pausing before cleanup of step 'StepRun'. Press enter to continue.
==> dockerwsc: Killing the container: c962a66a011043520e4faa24ddeb2f6bd5ba22d3937f763a5063585225aa4143
==> dockerwsc: Pausing before cleanup of step 'StepPull'. Press enter to continue.
==> dockerwsc: Pausing before cleanup of step 'StepTempDir'. Press enter to continue.
Build 'dockerwsc' errored: Error processing command: Error uploading ps script containing env vars: Failed to upload to '${env:SYSTEMROOT}\Temp\packer-env-vars-5aa0f937-7425-d5f0-30cf-3b48517f1606.ps1' in container: destination "c962a66a011043520e4faa24ddeb2f6bd5ba22d3937f763a5063585225aa4143:${env:SYSTEMROOT}\Temp" must be a directory
. exit status 1.

The packer json really is very simple but any provisioner (not just powershell) breaks it. Tried windows-shell too with similar error.

Note PACKER_TMP_DIR is set in my environment - I had temp container volume issues until I did that.

set PACKER_TMP_DIR=c:\tmp

JSON:
{
"builders": [
{
"name": "dockerwsc",
"type": "docker",
"container_dir": "c:/packerfiles",
"volumes": [{
"c:/Users/simon/packer/packer_templates/hypv-and-docker-win2012r2/windows/common": "c:/common"
}],
"image": "microsoft/windowsservercore",
"run_command": ["-dit", "{{.Image}}", "powershell"],
"export_path": "image.tar"
}
],
"provisioners": [
{
"type": "powershell",
"inline": [
"Write-Host Hello!"
]
}]
}

Debug extract:
2018/03/08 09:56:53 ui: ==> dockerwsc: Provisioning with Powershell...
==> dockerwsc: Provisioning with Powershell...
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Found command: Write-Host Hello!
2018/03/08 09:56:53 ui: ==> dockerwsc: Provisioning with powershell script: C:\Users\simon\AppData\Local\Temp\packer-powershell-provisioner311591131
==> dockerwsc: Provisioning with powershell script: C:\Users\simon\AppData\Local\Temp\packer-powershell-provisioner311591131
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Opening C:\Users\simon\AppData\Local\Temp\packer-powershell-provisioner311591131 for reading
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Uploading env vars to ${env:SYSTEMROOT}\Temp\packer-env-vars-5aa108e5-e015-6760-884d-80f5fcb5540b.ps1
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 [INFO] 71 bytes written for 'uploadData'
2018/03/08 09:56:53 [INFO] 71 bytes written for 'uploadData'
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Executing docker exec -i f5b520360ca08e8ad3da0af52ca61d6e4cc3512a90a3c0f96f0476e36f6974f2 /bin/sh -c (test -d "${env:SYSTEMROOT}\Temp\
packer-env-vars-5aa108e5-e015-6760-884d-80f5fcb5540b.ps1"):
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Copying to ${env:SYSTEMROOT}\Temp\packer-env-vars-5aa108e5-e015-6760-884d-80f5fcb5540b.ps1 on container f5b520360ca08e8ad3da0af52ca61d
6e4cc3512a90a3c0f96f0476e36f6974f2.
2018/03/08 09:56:53 packer.exe: 2018/03/08 09:56:53 Copied 71 bytes for ${env:SYSTEMROOT}\Temp\packer-env-vars-5aa108e5-e015-6760-884d-80f5fcb5540b.ps1
2018/03/08 09:56:53 [INFO] (telemetry) ending powershell
2018/03/08 09:56:53 ui: ask: ==> dockerwsc: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
==> dockerwsc: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
2018/03/08 11:14:04 ui: ask: ==> dockerwsc: Pausing before cleanup of step 'StepRun'. Press enter to continue.
==> dockerwsc: Pausing before cleanup of step 'StepRun'. Press enter to continue.

Why is packer instructing docker to execute command docker exec -i f5b... /bin/sh -c (test -d "${env:SYSTEMROOT}\Temp....

/bin/sh is not present in windows and nor is the test command. Is this the real issue?

@slime-uk I think it's a very similar issue (likely caused by the same PR) but the fix is actually in a different part of the code, in the docker communicator rather than the SSH communicator. We definitely have two parallel issues being investigated here:

1) As best I can tell, the slashes going the wrong way within the powershell provisioner cause failures for people using cygwin or other unix-style-pathing on windows. It looks like @DanHam has done a great deep dive to figure out root causes here, since we should be correcting any slash weirdness within the upload/download code. The bug on the user side should already be fixed by the binary I linked here and I'll investigate the deeper issue Dan found later on.

2) some code I added to standardize Packer behavior around uploading files to directories contains bad assumptions around the operating system of the guest in the ssh and docker communicators. I'll try to set up an environment for testing 2) today, and will upload a binary for you to test when I've gotten the code figured out. Note that this code shouldn't explicitly break anything unless you are uploading a directory and your pathing is weird, and in your case after this test runs, Packer still tries to do its normal thing of uploading your file into the desired location. So I don't actually think that this is the root of any of the issues here, although it is confusing and means that the behavior of the code probably isn't exactly what I intended. Try the linked binary that solved 1) and let me know if that solves your issue.

Yeah, just to be extra clear on this, I'm pretty sure @slime-uk 's bug is the same as @lbergnehr 's and should be fixed by the second binary I uploaded, or by freshly building the #5973 PR yourself.

The test -d thing is a red herring for all of the problems reported in this thread, but should definitely be investigated.

I've created a new issue, https://github.com/hashicorp/packer/issues/5994, to track the test -d thing.

@SwampDragons Hi. There are a few more paths to fix - they're hiding right down the bottom of the provisioner.go file!!

These are the ones that are giving @lbergnehr problems as they are the upload path for the elevated powershell script. You've already fixed the upload paths for the non-elevated powershell script with the changes made in #5973 - as we saw, these fixed the problem @ramesh45345 was having.

Just need to fix these paths and add the changes to your PR before you merge: HERE and HERE.

I'm not sure the logFile path was causing an issue, as this is used internally in template for the Windows task that runs the script... however, it can't hurt to be consistent!

Speaking of consistency, I wondered if we should change the paths mentioned above to use System Context environment variables rather than User Context variables. Clearly, things work as they stand, but consistency is nice!

func (p *Provisioner) generateElevatedRunner(command string) (uploadedPath string, err error) {
    ...
    path := fmt.Sprintf(`${env:SYSTEMROOT}/Temp/packer-elevated-shell-%s.ps1`, uuid)
    ...
    path = fmt.Sprintf("%s-%s.ps1", "%SYSTEMROOT%/Temp/packer-elevated-shell", uuid)
    ...
}

As you've mentioned, the changes made will fix things for now. However, they don't fix the root cause/issue with the communicator code. I think we should probably open an issue to look at the handling of paths by the communicator as I suspect it will only be a matter of time before a backslash style path works it's way back into the code somewhere and starts causing trouble again! My feeling is that the communicator should gracefully handle whatever style paths are thrown at it. I think I'd be happy to take a look at getting a fix in place for that...

@DanHam good catch, and agreed; I'll fix the remaining paths in the powershell provisioner now. You wanna do the honors of opening the root cause issue?

Here's the final dev binary built for Linux:
packer.tar.gz
And built for OSX:
packer.tar.gz

Thanks for the information. The packer fix is for Linux. I’m on Windows 10 so tried to build but problems launching docker container to build (Corp GPO on windows firewall disallowing me to share volume with docker Linux container). Will look more ASAP.

@slime-uk Ah, my bad! Here's a windows build for you :) packer.zip

Thanks. Sadly the fix appears to have made no difference for me. When packer launches the windows docker container and you specify powershell as the provisioner, it uploads some environment vars by default. It is this step that is failing (still):

`
C:\Users\simon\packer\packer_templates\hypv-and-docker-win2012r2>packer build docker-wsc-psinline.json
dockerwsc output will be in this color.

==> dockerwsc: Creating a temporary directory for sharing data...
==> dockerwsc: Pulling Docker image: microsoft/windowsservercore
dockerwsc: Using default tag: latest
dockerwsc: latest: Pulling from microsoft/windowsservercore
dockerwsc: Digest: sha256:554b69722f31381f10a2f3ddd81e4cb50beba7af8561e9014a9e431a52c8f825
dockerwsc: Status: Image is up to date for microsoft/windowsservercore:latest
==> dockerwsc: Starting docker container...
dockerwsc: Run command: docker run -v c:/tmp/packer-docker079487207:c:/packerfiles -dit microsoft/windowsservercore powershell
dockerwsc: Container ID: ed94f073578a21b92cb70dd9bf7f3257bc4737b37e10dc118d1da70f2a53fdcd
==> dockerwsc: Provisioning with Powershell...
==> dockerwsc: Provisioning with powershell script: C:\Users\simon\AppData\Local\Temp\packer-powershell-provisioner011644399
==> dockerwsc: Killing the container: ed94f073578a21b92cb70dd9bf7f3257bc4737b37e10dc118d1da70f2a53fdcd
Build 'dockerwsc' errored: Error processing command: Error uploading ps script containing env vars: Failed to upload to '${env:SYSTEMROOT}/Temp/packer-env-vars-5aa64763-89cb-2aec-2af7-e7e902001c37.ps1' in container: destination "ed94f073578a21b92cb70dd9bf7f3257bc4737b37e10dc118d1da70f2a53fdcd:${env:SYSTEMROOT}\Temp" must be a directory
. exit status 1.

==> Some builds didn't complete successfully and had errors:
--> dockerwsc: Error processing command: Error uploading ps script containing env vars: Failed to upload to '${env:SYSTEMROOT}/Temp/packer-env-vars-5aa64763-89cb-2aec-2af7-e7e902001c37.ps1' in container: destination "ed94f073578a21b92cb70dd9bf7f3257bc4737b37e10dc118d1da70f2a53fdcd:${env:SYSTEMROOT}\Temp" must be a directory
. exit status 1.

==> Builds finished but no artifacts were created.
`

@skime-uk I think you were right to open a separate issue. I think your problems are very likely more related to docker than to this ticket. We'll follow up with you in #6006

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

Was this page helpful?
0 / 5 - 0 ratings