Packer: file provisioner follows symlinks

[discordianfish]

Hi,

the file provisioner seems to always follow symlinks.

Given a provisioner like this:

   {
      "type": "file",
      "source": "files",
      "destination": "/test"
    }

and a file files/bar and a symlink files/foo -> files/bar, packer will follow the symlink so you end up with two regular files /test.

Instead, packer should copy the symlinks as they are. You can argue that there might be valid scenarios for following symlinks, so maybe making this an option is an alternative.
If things should be kept simple from a UX perspective, packer could also only follow symlinks if they point outside the source directory or something like that.

[xbglowx]

+1

[cbednarski]

Two reason we don't copy symlinks are because we don't know in advance if the target filesystem supports them (it could be Windows, for example), and we don't know if the thing they're pointing at exists on the remote side either. Paths are copied as streams much the same way they would be using scp to guarantee they show up inside the VM.

[discordianfish]

@cbednarski
1) It could fail hard on Windows and leave to the user to not include symlinks. Beside that, how many people are using packer for Windows?
2) it's up to the user to make sure the target exists. That's what people expect IMO.

I think in general it's very common to have symlinks in files you add to a system while provisioning. In my current case it's runit sv -> service symlinks but I would say that almost every time I needed to provision systems in one way or the other I used symlinks.

[deitch]

I just stumbled across this as well. In my case, I had symlinks to a target that did not exist, but would later, and the entire provision failed.

I also heavily use symlinks when customizing a systems-controlled system, so these fail as well.

[deitch]

And I thought I was smart. Instead of uploading the directory, I will use a shell-local provisioner to tar.gz the directory up, upload the tar.gz file using a file provisioner, and then extract using shell provisioner.

No such luck. file provision fails if the file doesn't exist at start!

So how do you get symlinks loaded up to the built image? I cannot use file upload with directory, because it has symlinks; and I cannot use shell-local to build a tar.gz and upload it, because the file doesn't exist yet!

[nullify005]

Would be great to have even just a hint of documentation on this at https://www.packer.io/docs/provisioners/file.html so that users are aware of the intended behaviour

[ryangraham]

This is problematic. Thanks @deitch you saved me some time exploring the local shell tarball option.

[deitch]

@ryangraham quite welcome. I ended up using a wrapping script that sets it all up. Suboptimal, but work with the tools we have. :-)

[mwhooker]

Thanks for the feedback. Right now I don't think we're going to add an option for choosing how to follow symbolic links. I've updated the documentation to clarify the behavior and demonstrate a workaround.

[discordianfish]

Obviously this is up to you but I'd really encourage you to re-evaluate this decision. Every few months I run into this issue and the worarkounds of wrapping everything in a shell script or create a tarball first is really annoying. Symlinks are so ubiquitous, even Windows 10 appears to support symlinks now.

I understand that a symlink isn't portable which I believe is a reason why you don't want to support this. But adding a flag to copy symlinks as-is that fails cleanly on systems where this isn't support seems really straight forward to me.

[discordianfish]

@mwhooker Also; did you end up changing the documentation? Because I can't find anything in there.

[mwhooker]

yup, it's in the PR above, but likely not on live yet https://github.com/mitchellh/packer/pull/4473

Always open to reevaluating things. I think If you want to see this behavior change, the best way would be to open an RFC ticket so we can plan what the new behavior would look like, how it would work across different OSes, and what would need to change in the code.