Packer: Vagrant Post-Processor on vmware-iso remote ESXi fails due to lstat no such file or directory
Using the vmware-iso build with remote_type esx5, the vagrant post processor step fails which causes the whole build to fail.
Build 'vmware-iso' errored: lstat output-macos1012-vmware-iso: no such file or directory
2016/10/19 21:03:38 ui error: Build 'vmware-iso' errored: lstat output-macos1012-vmware-iso: no such file or directory
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: lstat output-macos1012-vmware-iso: no such file or directory
==> Builds finished but no artifacts were created.
2016/10/19 21:03:38 Builds completed. Waiting on interrupt barrier...
2016/10/19 21:03:38 machine readable: error-count []string{"1"}
2016/10/19 21:03:38 ui error:
==> Some builds didn't complete successfully and had errors:
2016/10/19 21:03:38 machine readable: vmware-iso,error []string{"lstat output-macos1012-vmware-iso: no such file or directory"}
2016/10/19 21:03:38 ui error: --> vmware-iso: lstat output-macos1012-vmware-iso: no such file or directory
2016/10/19 21:03:38 ui:
==> Builds finished but no artifacts were created.
It seems probably due to the "keep_input_artifact": false, the directory is being deleted, but it seems somewhere in the code it tries to delete it a second time which causes the errors as the files have already been deleted. The logs don't give stack traces so it's hard for me to work out where it's going wrong exactly.
With "keep_input_artifact": true, I get the same error.
- Packer version: v0.10.2, and v0.11.0.dev with local patch to resolve #4022
- Host Platform: Linux / ESXi 5.5
- Debug log output from
PACKER_LOG=1 packer build template.json.
https://gist.github.com/berney/db4af355bafb80437fa81dbf42d9a24a - The _simplest example template and scripts_ needed to reproduce the bug.
Configs are based off https://github.com/boxcutter/macos with some edits for remote ESXi building.
Same config as reported in #4022, but with the vagrant processor
https://gist.github.com/berney/0dd666256ea2f8cf030a65109d3360f0
This is with the vagrant post-processor stanza: -
"post-processors": [
{
"keep_input_artifact": false,
"output": "box/{{.Provider}}/{{user `vm_name`}}-{{user `cm`}}{{user `cm_version`}}-{{user `version`}}.box",
"type": "vagrant",
"vagrantfile_template": "{{ user `vagrantfile_template` }}"
}
],
It's what comes from the boxcutter/macos repo, in #4022 this stanza was removed due to this issue, which is why it's missing in the gist.
berney
All 13 comments
This might be related, I didn't see it when I searched before opening a new bug: #4055
berney
on 25 Oct 2016
OK I've been trying to get to the bottom of this and I can no longer find what I did with the logs from the "successful" build after I had disabled the vagrant post-processor. I'm now doubt it was successful. I think I was judging success by the fact that the output directory still existed.
I've iteratively built debug versions of packer to work out what is happening, and I believe the error lies here: -
diff --git a/builder/vmware/iso/builder.go b/builder/vmware/iso/builder.go
index b70f6b8..e4bbde2 100755
--- a/builder/vmware/iso/builder.go
+++ b/builder/vmware/iso/builder.go
@@ -330,14 +335,18 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
// Compile the artifact list
var files []string
- if b.config.RemoteType != "" {
+ if b.config.RemoteType == "" {
dir = new(vmwcommon.LocalOutputDir)
dir.SetOutputDir(b.config.OutputDir)
files, err = dir.ListFiles()
} else {
files, err = state.Get("dir").(OutputDir).ListFiles()
}
IIUC originally would look at the local filesystem for the VM files that were built on the ESXi server, generating the lstat output-macos1012-vmware-iso: no such file or directory error, which is wrong.
After inverting the logic it will use the state/drivers dir interface to list files, which IIUC should work for local and ESXi builds. I no longer get the lstat error when doing a non-vagrant-post-processor build, and I see Build 'vmware-iso' finished, The artifacts of the successful builds are:, VM files in directory:, etc. messages. Looks good to me.
With a vagrant post-processor I get it trying to rm -rf the output directory twice, and getting an error on the second attempt as the files have already been deleted.
berney
on 4 Nov 2016
The above b.config.RemoteType issue I hit was fixed in v0.11.0, as there's an extra condition to the if added in #3050. https://github.com/mitchellh/packer/commit/23246c01cfb402c3e32b3d43ecaded42ccf17fc0.
I tested on some dev version of "0.11" based off master that was before this patch.
I'll have to test v0.11.0 release, and confirm this issue is resolved.
Testing on the master v0.10.2 with patches to invert the if condition above works when there's no vagrant post-processor but fails when there is a post-processor. I'm testing if the latest stable build fixes this issue.
berney
on 4 Nov 2016
I'm still getting errors after trying v0.11.0 and master (adding my patch to work-around #4022, https://github.com/mitchellh/packer/issues/4022#issuecomment-254471251).
I've added some debugging logs to packer and I'm trying to narrow down where the issue is. I'll upload logs of the errors today.
berney
on 8 Nov 2016
OK I believe the cause of the issue is that the vagrant post-processor is looking at the local file system rather than using the remote file system (ESXi).
post-processor/vagrant/post-processor.go
func (p *PostProcessor) PostProcessProvider(name string, provider Provider, ui packer.Ui, artifact packer.Artifact) (packer.Artifact, bool, error) {
// ... snip ...
// Run the provider processing step
vagrantfile, metadata, err := provider.Process(ui, artifact, dir)
if err != nil {
return nil, false, err
}
post-processor/vagrant/vmware.go
func (p *VMwareProvider) Process(ui packer.Ui, artifact packer.Artifact, dir string) (vagrantfile string, metadata map[string]interface{}, err error) {
// Create the metadata
metadata = map[string]interface{}{"provider": "vmware_desktop"}
// Copy all of the original contents into the temporary directory
for _, path := range artifact.Files() {
ui.Message(fmt.Sprintf("Copying: %s", path))
dstPath := filepath.Join(dir, filepath.Base(path))
if err = CopyContents(dstPath, path); err != nil {
return
}
}
return
}
post-processor/vagrant/util.go
// Copies a file by copying the contents of the file to another place.
func CopyContents(dst, src string) error {
srcF, err := os.Open(src)
if err != nil {
return err
}
defer srcF.Close()
dstDir, _ := filepath.Split(dst)
if dstDir != "" {
err := os.MkdirAll(dstDir, os.ModePerm)
if err != nil {
return err
}
}
dstF, err := os.Create(dst)
if err != nil {
return err
}
defer dstF.Close()
if _, err := io.Copy(dstF, srcF); err != nil {
return err
}
return nil
}
In the logs, right after I see vmware-iso (vagrant): Copying: /vmfs/volumes/datastore1/output-macos1012-vmware-iso/disk-flat.vmdk, I see that provider.Process() got an error, the next step is Deleting original artifact for build 'vmware-iso' and it then SSHs in and rm -rf the directory (/vmfs/volumes/datastore1/output-macos1012-vmware-iso/disk-flat.vmdk). I did one more test to confirm that the original error is occurring in this bit of code: -
post-processor/vagrant/util.go
// Copies a file by copying the contents of the file to another place.
func CopyContents(dst, src string) error {
srcF, err := os.Open(src)
if err != nil {
return err
}
I added a log message, and I can see that this is the where the error occurs. The vagrant post-processor doesn't understand how to handle ESXi remote builds and tries to copy from the local file system, which is wrong, os.Open in CopyContents errors, which bubbles up through vmware::Process() and vagrant::PostProcessProvider(), after which Artifact::Destroy() called which rm -rfs the directory where the VM and the build fails.
berney
on 8 Nov 2016
@rickard-von-essen I think you should remove waiting-reply label as I have provided more info and add build-failure back as my investigation shows that the vagrant post-processor doesn't properly handle files on the remote ESXi server's filesystem, it incorrectly treats them as local files causing the build to fail with an error.
berney
on 14 Dec 2016
I'm still seeing this happen in 0.12.3, exactly as @berney described. Any updates since December?
sagargp
on 17 Feb 2017
+1 I think I am seeing this issue as well. 0.12.2 - OSX.
Using Packer with the "vmware-iso" builder type, that works successfully. However, it looks like it shuts down the VM and "Clean up" before running any post processors. In which in my case is vagrant.
````
vmware-iso output will be in this color.
==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///14393.0.160715-1616.RS1_RELEASE_CLIENTENTERPRISEEVAL_OEMRET_X64FRE_EN-US.ISO
vmware-iso: Error downloading: stat /14393.0.160715-1616.RS1_RELEASE_CLIENTENTERPRISEEVAL_OEMRET_X64FRE_EN-US.ISO: no such file or directory
vmware-iso: Downloading or copying: http://care.dlservice.microsoft.com/dl/download/2/5/4/254230E8-AEA5-43C5-94F6-88CE222A5846/14393.0.160715-1616.RS1_RELEASE_CLIENTENTERPRISEEVAL_OEMRET_X64FRE_EN-US.ISO
==> vmware-iso: Creating floppy disk...
vmware-iso: Copying files flatly from floppy_files
vmware-iso: Copying file: /Users/gl/Library/Caches/malboxes/Autounattend.xml
vmware-iso: Copying file: /usr/local/lib/python3.6/site-packages/malboxes-0.3.0.dev0-py3.6.egg/malboxes/installconfig/windows10_64/enablewinrm.ps1
vmware-iso: Done copying files from floppy_files
vmware-iso: Collecting paths from floppy_dirs
vmware-iso: Resulting paths from floppy_dirs : []
vmware-iso: Done copying paths from floppy_dirs
==> vmware-iso: Uploading Floppy to remote machine...
==> vmware-iso: Uploading ISO to remote machine...
==> vmware-iso: Creating virtual machine disk
==> vmware-iso: Building and writing VMX file
==> vmware-iso: Registering remote VM...
==> vmware-iso: Starting virtual machine...
==> vmware-iso: Waiting 10s for boot...
==> vmware-iso: Connecting to VM via VNC
==> vmware-iso: Typing the boot command over VNC...
==> vmware-iso: Waiting for WinRM to become available...
==> vmware-iso: Connected to WinRM!
==> vmware-iso: Gracefully halting virtual machine...
vmware-iso: Waiting for VMware to clean up after itself...
==> vmware-iso: Deleting unnecessary VMware files...
vmware-iso: Deleting: /vmfs/volumes/datastore2/builds/vmware.log
==> vmware-iso: Compacting the disk image
==> vmware-iso: Cleaning VMX prior to finishing up...
vmware-iso: Unmounting floppy from VMX...
vmware-iso: Detaching ISO from CD-ROM device...
vmware-iso: Disabling VNC server...
==> vmware-iso: Keeping virtual machine registered with ESX host (keep_registered = true)
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for 'vmware' provider
vmware-iso (vagrant): Copying: /vmfs/volumes/datastore2/builds/packer-test.nvram
Build 'vmware-iso' errored: 1 error(s) occurred:
- Post-processor failed: open /vmfs/volumes/datastore2/builds/packer-test.nvram: no such file or directory
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: 1 error(s) occurred:
- Post-processor failed: open /vmfs/volumes/datastore2/builds/packer-test.nvram: no such file or directory
==> Builds finished but no artifacts were created.
packer completed with return code: 1
Packer failed. Build failed. Exiting...
```
gleblanc1783
on 24 Feb 2017
@gleblanc1783 can you share your .json template?
lmayorga1980
on 8 Mar 2017
I'd also like to see it.
It doesn't try to install VMware tools? I had hit another/different issue there #4022.
berney
on 9 Mar 2017
I still receiving the same error ...nvram: no such file or directory with packer version 1.1.3. I try to build on ESXi 6.5. Debug output:
2018/01/17 15:12:55 packer: 2018/01/17 15:12:55 starting remote command: ls -1p /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229
2018/01/17 15:12:55 [INFO] (telemetry) ending vmware-iso
2018/01/17 15:12:55 ui: ==> vmware-iso: Running post-processor: vagrant
2018/01/17 15:12:55 [INFO] (telemetry) Starting post-processor vagrant
==> vmware-iso: Running post-processor: vagrant
2018/01/17 15:12:55 ui: ==> vmware-iso (vagrant): Creating Vagrant box for 'vmware' provider
==> vmware-iso (vagrant): Creating Vagrant box for 'vmware' provider
2018/01/17 15:12:55 ui: vmware-iso (vagrant): Copying: /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram
vmware-iso (vagrant): Copying: /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram
2018/01/17 15:12:55 [INFO] (telemetry) ending vagrant
2018/01/17 15:12:55 [INFO] (telemetry) found error: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory
2018/01/17 15:12:55 Deleting original artifact for build 'vmware-iso'
2018/01/17 15:12:55 packer: 2018/01/17 15:12:55 opening new ssh session
2018/01/17 15:12:55 packer: 2018/01/17 15:12:55 starting remote command: rm -rf /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229
Build 'vmware-iso' errored: 1 error(s) occurred:
* Post-processor failed: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: 1 error(s) occurred:
2018/01/17 15:12:56 ui error: Build 'vmware-iso' errored: 1 error(s) occurred:
* Post-processor failed: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory
2018/01/17 15:12:56 Builds completed. Waiting on interrupt barrier...
2018/01/17 15:12:56 machine readable: error-count []string{"1"}
2018/01/17 15:12:56 ui error:
==> Some builds didn't complete successfully and had errors:
2018/01/17 15:12:56 machine readable: vmware-iso,error []string{"1 error(s) occurred:\n\n* Post-processor failed: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory"}
2018/01/17 15:12:56 ui error: --> vmware-iso: 1 error(s) occurred:
* Post-processor failed: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory
2018/01/17 15:12:56 ui:
==> Builds finished but no artifacts were created.
2018/01/17 15:12:56 [INFO] (telemetry) Finalizing.
* Post-processor failed: open /vmfs/volumes/local/NetBSD-7.1-amd64-minimal-v7.1.20171229/NetBSD-7.1-amd64-minimal-v7.1.20171229.nvram: no such file or directory
==> Builds finished but no artifacts were created.
Used packer template:
https://gist.github.com/drscream/0b871b030008ff8e8c8a3c8ae7640eb8
drscream
on 17 Jan 2018
I'm going through old esxi issues right now to determine what to close and what to work on in my next esxi push. Reading over this one, it sounds like the issue is that folks wanted the vagrant post-processor to use the remote artifact to create the vagrant vm. The vagrant post-processor was definitely only written to work on a local filesystem, so I think I'd say this is expected behavior. I'll add some documentation that you need to export the vm from esxi to your local machine if you want to package it into a vagrant box.
If I've misunderstood the issue, let me know and I can reopen :)
SwampDragons
on 20 Apr 2019
I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.
![hashibot[bot] picture](https://avatars2.githubusercontent.com/in/8332?v=4&s=40)
hashibot[bot]
on 29 Mar 2020
Related issues
mwhooker
·
3Comments
mushon4
·
3Comments
PartyImp
·
3Comments
mvermaes
·
3Comments
Tensho
·
3Comments
Most helpful comment
I'm still seeing this happen in 0.12.3, exactly as @berney described. Any updates since December?