Packer: [Feature] When retrying `on-error`, use latest provisioner data from template

[glasnt]

Using -on-error=ask, selecting [r]etry iterates again through all the provisioners.

Depending on the provisioners (in my case, ansible-local), part of the provisioning step is to re-upload associated files, so iterative testing within the scope of that is possible. However, any changes made to the provisioner node itself is ignored.

I was trying to retry iterative changes in my template file, but not seeing any changes on the system, until I realised that changes to the template file weren't being seen.

Feature request: re-load the provisioners from the template file when retrying on-error.

This will allow for faster iterative testing/debugging of templates (without having to wait for termination/reprovisioning)

[SwampDragons]

So to clarify, you're changing the packer template itself, and want the template to be reloaded between retry runs?

This is a really interesting idea and I can see the value in it, but it's definitely not how the original authors expected things to be debugged. Because of the way we generate the template and use it internally, my gut says this would be a fairly difficult change to make. But I like the idea so I'll take a look and see what I can think of.

[glasnt]

Hi @SwampDragons, yup, that'd be what I'm after.

If this is not possible, just sufficient documentation in the section on retry detailing that the template is not reloaded would be helpful. I understand that different provisioners would work in different ways, so just detailing what packer proper does would be useful to me. If you accept doc update PRs like this, just point me where it belongs and I can create a PR

[SwampDragons]

I love docs contributions; this probably belongs in https://github.com/hashicorp/packer/blob/master/website/source/docs/other/debugging.html.md

I'm waist-deep in final testing for our next release, so I won't get to this myself for a little while, but I'll take a look at clarifying docs and/or figuring out if we can reload provisioners when I get a chance.

[glasnt]

6679 creates a documentation note to clarify current functionality. Totally okay for this Feature Request to be closed if the doc PR is merged.

[SwampDragons]

I've set this for the v 1.3.1 milestone not necessarily because I think we can do it by then but because I'd like to make sure that the option gets investigated by then.

[SwampDragons]

I took a look at the code for this and it would be a fairly large change.

We'd need to make sure we can find the original template filename inside of this loop and then would basically have to copy this code into the retry portion of that loop; currently the template filename isn't even stored as a variable; it's just args[0] in build.go and then the template object is used for the rest of the build. I think we'd have to attach the file name to the template in order to access it later.

We'd also have to do some kind of step number tracking, which we don't currently do. We'd need to have the basic step runner track what step it's on, pass that information into the step's call to run, and then allow the step to re-parse the template, select itself based on step number, and rerun.

But when that template reload happens, nothing outside of the current step itself would be modified when the template got reloaded from within a step, so following steps would still use the old template unless they got retried as well. I'm worried this is going to cause a confusing interface for users. We don't really have a way of guaranteeing, for example, that a user doesn't add a new step to the template and throw off our indexing. Maybe this is obvious and wouldn't come up, but I'm worried that once we open the door to allowing template reloads and modifications on the fly, user expectations of how Packer should reload the template will be wildly disparate.

I think for now I'm going to close this issue, though I'm sad about it because I really do like the idea. I just don't see a super clean implementation/ui.

[glasnt]

Thank you for detailing exactly where and how this issue happens. I really do appreciate the effort, especially since it can be used as reference later to this limitation.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.