Terraform: File provisioner - upload of directory fails when it contains a symlink

[bpoland]

Terraform Version

Terraform v0.9.2 (regression from Terraform 0.8.8 and earlier where this worked)

Affected Resource(s)

File provisioner

Terraform Configuration Files

Tested with multiple providers like vsphere and azurerm and was able to consistently reproduce with both. I put two file provisioners, the first uploading a single file and the second uploading a folder:

provisioner "file" {
    source = "${path.module}/file"
    destination = "/tmp/file"
}
provisioner "file" {
    source = "${path.module}/folder"
    destination = "/tmp"
}

Debug Output

https://gist.github.com/bpoland/278bdcd772712ec750fd874b8e844767

Panic Output

no panic

Expected Behavior

Both the file and the folder should have been uploaded to the deployed VM without errors.

Actual Behavior

1 error(s) occurred:

• vsphere_virtual_machine.vm: 1 error(s) occurred:

• Upload failed: Process exited with status 1

Steps to Reproduce

1. Create a folder on disk with a file inside it, and a second file symlinking to the first, e.g.
   mkdir folder
   touch folder/fileinfolder
   ln -s folder/fileinfolder folder/linkinfolder

2. Add a file provisioner to your VM resource, like the above, which should upload the "folder" folder to your VM after creating it

3. terraform apply

4. When terraform tries to upload the folder, it will fail with the above error.

Important Factoids

• I tried using Terraform 0.8.8 against the exact same .tf file and it worked perfectly. Switching to terraform 0.9 or later causes this issue.
• After the failure, the symlink file does end up in the target VM but it's a regular file not a symlink and only has some content in it, not the full content of the file that it was linked to.

References

none

[msackman]

Is there any way of telling if this is going to be addressed in 0.9.3?
EDIT: Or indeed 0.9.4 now that 0.9.3 is out and the changelog doesn't make any suggestion this has been fixed.

[cursork]

My project hit this too

[Ferada]

"Me too", repeatedly; I've also looked at the implementation of the SSH communicator module, if someone has a suggestion as to how to debug that - right now I'm not seeing any meaningful error from the SSH transport, especially since scp -rvt only returns an -1 exit status without much further explanation.

[rbposborne]

I'm also hitting this issue. A real pain.

[mantiz]

I'm also hitting this issue with version 0.10.0. :-/

[maage]

I have following temporary fix for this regression. You need to have rsync and ssh installed. You might get nasty warnings about host keys. Also there is some assuptions about ssh key locations and variables.

original code:

  connection {
    user = "ec2-user"
    host = "${openstack_compute_instance_v2.instance.access_ip_v4}"
  }

  provisioner "file" {
    source      = "${path.module}/setup"
    destination = "."
  }

temporary fix:

  provisioner "local-exec" {
    command = "rsync -avLe 'ssh -ax -i ${path.module}/../../ssh_keys/${var.keypair}.pub -S none -o BatchMode=yes -o StrictHostKeyChecking=no' ${path.module}/setup ec2-user@${openstack_compute_instance_v2.instance.access_ip_v4}:."
  }

[eigood]

This is still happening with 0.12.23; if I remove the single symlink in my tree, then the upload finishes. There are 2679 non-symlinked files(saw another issue that said >2000 files would fail).