Packer: Don't copy when using a local ISO

[dglinder]

When building from a local ISO (e.g. "C:tempisosmy_os.iso"), don't do an additional "Downloading or copying" of the ISO file. Even on a local SSD, copying a 5+GB DVD ISO to another location on the same drive adds unnecessary time to build. A simple flag in the JSON file such as "iso_use_in_place" could be added to remove the "local vs remote" decision from the program.

[vtolstov]

I thinks that we need to detect via Stat file presence and if it exists use it as-is

[arizvisa]

Ftr, the downloader currently supports this using the CopyFile field in DownloadConfig which is set to False in common/step_download.go. So this default value should just need to be exposed to the json template using some agreed-upon name.

[azr]

I think the iso_inplace is a good option, will start working on this as my first issue if we all agree :slightly_smiling_face:

[SwampDragons]

@dglinder Can you confirm that PR #6593 solves this for you? here's a windows binary of that branch:
packer.zip

[dglinder]

In the builders section of the JSON file I added "iso_inplace": "true", along side the iso_checksum and iso_url settings:

Unfortunately packer still copies it:

I did validate that I'm using the new binary, running with --version shows 1.2.6.

Did I mess up the syntax for the JSON file?

[azr]

Howdy @dglinder thanks for the feedback !

Your settings are correct and packer will show Downloading or copying ISO in any case;
that message is misleading.

Look for the following message when running in debug mode :

https://github.com/hashicorp/packer/blob/fae3db4e5879e868db526fc515d8167fcfc55fa8/common/download.go#L187

Versus:
https://github.com/hashicorp/packer/blob/fae3db4e5879e868db526fc515d8167fcfc55fa8/common/download.go#L172

May be I should address that message too...

Edit: I can't, the downloader doesn't seem to have the state bag which contains ui logger to show nice messages.

[dglinder]

I finally got back to this project - I can confirm that in Packer 1.3.0 it does use the file in-place. Thank you!

[martinandersson]

Excusez-moi, but if I choose a LOCAL FILE then how can anyone argue it is NOT "use in place"? The copy-behavior with or without extra flags and complexity makes no sense, or what am I missing?

[santhoshganesan]

could i have clear info about the iso_file_in_place variable for the packer verson 1.5.1

[martinandersson]

Documentation and proper log statements would be desirable. Even more desirable would be a rational application behavior.

Copy-in-place (note btw, 3 words.......) is exactly what I would expect the binary to do if I was pointing to a local ISO file. And if the ISO path is a mapped network drive or whatever then it is, so what? Let the file system manage that? "Premature optimization is the root of all evil".

How on earth can a whole bunch of developers just rush through this issue, "fixing it" by adding even more configuration? It's not the end user who should be required to implement "if-else" logic and force our app to do sensible things with bloated configuration. It is our app who allegedly is already written in a language that supports if-else statements who should. I mean seriously. Please correct me here if I am wrong here, I just have to be missing something, right? If not, please re-open this issue and do the right thing.

Actually, this issue should be re-opened regardless because the flag is as far as I can tell misspelled, undocumented and writes wrong statements in the UI. We could add docs something like this (warning, lots of irony):

You would think that a local ISO file is, well.. local. But no, the file will be copied over and over again into the cache directory. Yes, it will be duplicated each time you run a build because that's just what Packer loves to do; shred your disk space into oblivion and waste your personal time sitting around waiting on obnoxious disk operations. So, we've added this flag. If you set a local ISO path, you'll want to set this flag too! But hey, you should be used to this now. Half of our flags work the same way, you set flag X then you must set flag Y or have your entire build blow up. Welcome to the world of scripting and to the product of programmers who loves to layer code on top of code instead of doing the right thing! We follow the mantra; "never go back, just keep on adding".

[SwampDragons]

Hi Martin. I'd love some clarification here:

But no, the file will be copied over and over again into the cache directory. Yes, it will be duplicated each time you run a build

If you are experiencing this behavior, it's a bug. Packer should only copy files, whether local or from the internet, one time into the cache. It stores files using a hashing function on the URL, so it _should_ only re-cache if the path you provide changes.

If that is not the behavior you see, can you please open a new bug report with reproduction details so that the team can investigate what's going on?

Actually, this issue should be re-opened regardless because the flag is as far as I can tell misspelled, undocumented and writes wrong statements in the UI

This is not documented because we removed the flag three days after adding it, and it never made it into a release: https://github.com/hashicorp/packer/pull/6603/files. The issue turned out to be a confusing log line; at the time, we were already using files in place but the logs didn't reflect that.

In the time since this issue was closed, we've switched to using an external download library to try to reduce bugs and support more download protocols. But this required a few work-arounds; now, instead of just using the isos in place directly, we have to create a symlink from our cache to the path provided so that you can build without copying the iso.

Unfortunately, we discovered after integrating the new library that there were issues with Windows where the symlinking was breaking builds, which is why on Windows we currently copy the file into the cache instead of symlinking to it. (context: #7534 and #7575). We have a PR open against the download library we use, which will fix this so that we don't need to symlink at all. Once that PR gets merged, we'll be able to use files in-place for Windows again. We should still only be copying them into the cache once, though.

As a final note, I'd like to refer you to our community guidelines We all get frustrated, and I understand why you find this behavior frustrating. But in general, snark doesn't add to any professional conversation and has a chilling effect on community contributions, even when it's not intended personally.

If you have any more thoughts about defaults you think should be different, you're always welcome to open up feature requests. We don't know what our users want if you don't tell us, but in general commenting on long-closed issues isn't the most effective way to get features changed. Thanks!

[martinandersson]

SwampDragons, what a well-written and professional answer by you! I can say you lead by example, for sure =) And a super big thank you for the explanation. To me this was much needed. My apologies for letting my bafflement from reading this thread in isolation spill over into the "irony" parts of my text. My thought was that if a whole group of people can't see why adding yet more configuration on top of an already weird behavior is not the wrong approach then that irony was much needed. But there is no excuse, you are absolutely right. And SwampDragons, I kid you not, your answer was so professional I want to be like you.

Yes the file is duplicated, which is weird of course. I use Windows. None of my drives is on a network share. Both are SSD so not too different in terms of type either, I suppose. Well, if I understood correctly you will soon use a local file in-place regardless, so this problem shouldn't persist. If it do I will come back, with lots of irony!! hahahaha just kidding.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.