Packer: allow to serve http files containing variable(s)

[azr]

This has precedent: https://github.com/hashicorp/packer/issues/3961 https://github.com/hashicorp/packer/issues/1673 https://github.com/hashicorp/packer/issues/2867 https://github.com/hashicorp/packer/issues/4530 but was never exactly solved.

I would be nice if the contents served in HTTP by packer for the boot_command could be passed from a variable.

---

Why ?

Very often in packer buildfiles I see the boot command using local files, for example:

http_directory = "preseed_folder/"
boot_command   = [
    // ...
    " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg<wait>",
    // ...
]

Very often a preseed.cfg file for packer will contain the following settings:

d-i passwd/user-fullname string vagrant
d-i passwd/user-uid string 1000
d-i passwd/user-password password vagrant
d-i passwd/user-password-again password vagrant
d-i passwd/username string vagrant

This results in a lot of images actually having vagrant/vagrant as login. This is okay in most situations as a provisioning step will change this setting later on but making these actual variables could make the process a bit safer by may be passing ssh keys and faster by setting these credentials correctly initially.

[azr]

A proposal for this would be to add a new http_files setting that can be used like so:

http_files = {
  "some_file" = "contents"
  "preseed.cfg" = <<EOF
d-i passwd/user-fullname string ${var.user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${var.user_password}
d-i passwd/user-password-again password ${var.user_password}
d-i passwd/username string ${var.user_name}
EOF
# this could also be an ssh public key
}

This would start an http server that simply serves these string.
This would work in JSON or HCL2.

I'd keep it simple by not adding support for subfolders which seems like a 'nice-to-have' but I think it would be unnecessary.

---

For implementation:

I'd change these structs to have that new HTTPFiles map[string]string field:

https://github.com/hashicorp/packer/blob/d35f342319858e0059f9728e64f078af6a7a136b/common/http_config.go#L11-L37

https://github.com/hashicorp/packer/blob/d35f342319858e0059f9728e64f078af6a7a136b/common/step_http_server.go#L23-L30

One simple solution would be to serve them using a map[string]string:

func (m MyMap) handle(w http.ResponseWriter, r *http.Request) {
  file, found := m[r.URL.Path]
  // ....
}

Note that the map implementation makes the subfolder serving easy as we would just match the whole string.

Another solution ( which I like a bit less ) would be to create these files in a temporary folder and serve that folder.

[jhawk28]

Would it be simpler to use the golang template package (https://golang.org/pkg/text/template/) and serve up files with a particular extension?

[azr]

@jhawk28 I'm not sure to understand, text/template could be used to interpolate variables with the http_files I described before but it cannot serve these files.

[jhawk28]

@azr here is how I think it could work:

1. The user creates a preseed.cfg.pkrtemplate file
2. The packer.json file references {{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg
3. The http server sees the request for preseed.cfg and sees that there is a .pkrtemplate file for it. It executes the template with the variables and serves up the result.

[azr]

Ah I see, thanks for you suggestion, I like my suggestion a little bit better because it is a bit less magic and therefore easier to document & comprehend and if you were in HCL2 mode it would be super easy to put the template definition in a file on it's own, for example:

# preseed.cfg.pkr.hcl
locals {
  "preseed.cfg"  = <<EOF
d-i passwd/user-fullname string ${var.user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${var.user_password}
d-i passwd/user-password-again password ${var.user_password}
d-i passwd/username string ${var.user_name}
EOF
  }

[jhawk28]

to each his own. I prefer the ability to encapsulate the file outside of the packer json/hcl file.

[SwampDragons]

Why not save the templates off as their own files and have a map[string]string var named http_templates -- Packer could load the map keys, interpolate them as golang templates, and save them off as the map values inside the http_dir. Then it's neither "magically" loading files with a specific name nor forcing users to dump whole preseed files into variables.

[jhawk28]

I don't think you need to render the templates as files, you can just serve them up in the HTTP handler. We could just identify the templates in a list. If it allowed wildcards, it would reduce the config to allow the user to define something like *.pkrtemplate.

Example:

{
  "http_templates": ["preseed.cfg", "*.pkrtemplates"],
  "http_dir": "http"
}

[azr]

In HCL2 we could take the templatefile function from Terraform; this would allow to load template files in other places too, making the solution less specific and more general:

#preseed.cfg.tpl
d-i passwd/user-fullname string ${user_name}
d-i passwd/user-uid string 1000
d-i passwd/user-password password ${user_password}
d-i passwd/user-password-again password ${user_password}
d-i passwd/username string ${user_name}

# source.pkr.hcl
source "..." "..." {
  http_files = {
      "preseed.cfg" = templatefile("preseed.cfg.tpl", { user_name = var.user_name , user_password = var.user_password ),
  }
}

etc.

[radeksimko]

👋 Hello from the Terraform side,
Just wanted to note that if you do decide to adopt the templatefile approach which Terraform (and also Waypoint) uses, I'd highly recommend deciding on and documenting naming convention for these files.

This is important for syntax highlighting and editor integration in the future and ideally it should be Packer-specific, because that will allow future editor integrations to e.g. provide relevant completion for variables.

[azr]

Hey @radeksimko 👋🏼 ! Very important point yes, we will definitely keep that in mind, thanks for the heads up 🙂 .