Packer: PXE / Bare metal builder

[silverfisk]

An idea about a bare metal builder for packer. Perhaps it would be possible to somehow integrate a PXE server or configure an external one. Something like

{
"builders":
[
{
"type": "baremetal",
"iso_url": "http://mirror.centos.net/centos/6/isos/x86_64/CentOS-6.4-x86_64-minimal.iso",
"iso_checksum": "4a5fa01c81cc300f4729136e28ebe600",
"iso_checksum_type": "md5",
"HWaddr": "11:22:00:AA:AA:AA"
"boot_command":
[
"",
" ks=http://10.0.2.2:{{ .HTTPPort }}/centos6-ks.cfg"
]
}
]
}

[mitchellh]

It'd be interesting. I don't know enough about this to develop this myself, but if someone went ahead and made a POC then I'd take a look.

[mihai-satmarean]

This looks like this https://github.com/jedi4ever/veewee/issues/172 and it would really be useful for some folks like me.
I can help with testing if needed.
Thanks!

[justinclayton]

There seems to be some potential overlap with what razor is doing. I wonder if there could be some synergy here? The biggest hurdle I see is that razor is architected as a long-running web service and packer is not.

[mihai-satmarean]

Great point out, I had no Idea of Razor,
but as you mentioned, it is too much of an application with its own
prerequisites.
What I was wishing for was some IPMI capabilities from packer.io
I am willing to test on what hardware we have at our disposal E.G. FUJITSU
PRIMERGY RX200 S6, some Dell blades and Cisco UCS in the future.

2014-09-24 1:34 GMT+02:00 Justin Clayton [email protected]:

There seems to be some potential overlap with what razor
https://github.com/puppetlabs/razor-server is doing. I wonder if there
could be some synergy here? The biggest hurdle I see is that razor is
architected as a long-running web service and packer is not.

—
Reply to this email directly or view it on GitHub
https://github.com/mitchellh/packer/issues/955#issuecomment-56606170.

[vtolstov]

I'm use packer to create baremetal images. I'm use dracut and debian live-boot to bootup servers via pxe and download created by packer squashfs image.

[mihai-satmarean]

That is great news Vasiliy, can you post a link or an example?
That I would like to see.
Thanks!

2014-10-09 12:14 GMT+02:00 Vasiliy Tolstov [email protected]:

I'm use packer to create baremetal images. I'm use dracut and debian
live-boot to bootup servers via pxe and download created by packer squashfs
image.

—
Reply to this email directly or view it on GitHub
https://github.com/mitchellh/packer/issues/955#issuecomment-58489142.

[vtolstov]

=) pxe provided by ipxe.
to compress squashfs image i'm use own packer module https://github.com/vtolstov/packer-post-processor-squashfs

[mihai-satmarean]

I am asking, because lately I was playing with this
https://github.com/jpetazzo/pxe/network
but your idea sounds more close to what I want.
If I would only have time to test :)

PS:
Check the "MOAR fun" challenge at the end of the readme from the provided
link.
YOu'll love it

2014-10-09 15:34 GMT+02:00 Vasiliy Tolstov [email protected]:

=) pxe provided by ipxe.
to compress squashfs image i'm use own packer module
https://github.com/vtolstov/packer-post-processor-squashfs

—
Reply to this email directly or view it on GitHub
https://github.com/mitchellh/packer/issues/955#issuecomment-58509376.

[routelastresort]

:+1: , @vtolstov! This also might work as a USB thumbdrive installer for custom appliances (for the times when we're remote, and can't easily pxeboot. Cheers!

[vtolstov]

if libguestfs author rewrite golang bindings to supporting go get, i'm try to create pull request to import it into packer plugins.

[DavidJFelix]

+1 @vtolstov

[azimut]

+1

[richard-mauri]

I see this and related issues are closed.
We are tasked to use packer with bare metal so we can pre-bake like an Ubuntu 14.04 base machine image with custom packages on top, save to a repo like Artifactory and later have PXE be able to use that image. NO application package provisioning done at machine creation time; similar to how ami provisioning is done (at image build time).

Where does this stand today. I see there is some work by @vtolstov that is 2 years old.

It seems like there are many variables that must be factored out; hostname,locale,disk partitioning etc.

I wonder if I can have a packer builder create an iso image as its artifact?

[beddari]

@richard-mauri I got some experience with this and I'd say there is no real reason (at least that I can think of) to do the build step on a physical box. So in my opinion that leaves you with a lesser problem, pxeboot a RAM based image and deploy/write a ready-made image to the box.

If starting today I'd probably go with the The Foreman discovery image as a base.

[beddari]

That being said, also probably should mention a new project I found https://github.com/danderson/pixiecore ... which looks like it could form the basis of a baremetal builder.

[richard-mauri]

Thanks @beddari
I'll investigate your approaches, but it seems just a little off target.

Essentially, in high level terms, I want Packer to do a non GUI launch of like an Ubuntu Linux virtual machine (like it does with virtualbox-builder) and then take a path to an iso file that it loop mounts, copies to a writable location and then updates the writable location (adding packages) and then writing a new iso as the final output. Now, it seems that the bkisofs, which is at the heart of mkisofs and isomaster) can do the heavy lifting for the package addition updates.

Perhaps I have to write a go iso-builder that calls bk.

Again the goal is a Packer template that produces AMI, Vagrant/VBox box and ISO as we will use the first two in the dev phase and the latter in the PROD phase and want the same tooling to do all the provisioning.

The whole deal with PXE is a downstream concern that will consume the pre backed iso Produced by Packer. I don't need any automation there.

I hope that makes sense

[vtolstov]

I'm write my own scripts that run in provision after all things done.
One script creates tar file system, download file to host and use tar.gz via artifice to docker push
Second use mksquashfs to create squashfs that booting via pxe.
So i don't understand problems in this issue.
Create that you want inside vm, download and via artifice add this file to artifact list.

[zrml]

Any news on this? @richard-mauri: did you ever write anything for bare metal provisioning as you described?
thanks

[beddari]

If all you want to do is just input an iso, modify it, then output a new one ... @vtolstov is right on

1) Boot any VM
2) Download any iso inside that VM and modify it as needed, using any tooling you'd like
3) Use the artifice post-processor https://www.packer.io/docs/post-processors/artifice.html to override the artifact list, thus the build output will be your modified iso

In any case, that is not what this issue originally was about and I think I'm done, good luck :)

[openface]

@beddari this is exactly the workflow I'm looking to achieve with Packer. Do you know of any tutorials or snippets that would help a newcomer, such as myself, put this together?

[fxfitz]

@beddari @openface: I'm in the exact same boat. Any leads?

[viralpoetry]

@openface @fxfitz
This is quite old issue, but I partially solved this problem for my use case here:
https://github.com/viralpoetry/packer-bare-metal

Hope that it helps somebody.

[fxfitz]

Thanks @viralpoetry!

[saurabhvashisht]

@viralpoetry have you ever done the packer thing on windows 7 image or anyone else
have done it? I have a urgent need for this

[saurabhvashisht]

@vtolstov is it possible to share your script with me as I have very similar requirement

[vtolstov]

very simple: install all needed stuff inside vm (for debian live-boot, for fedora/centos dracut with ability to boot from net).
after sthat packer run script (example for debian):

#!/bin/sh -ex

apt-get -y install squashfs-tools

mkdir -p /mnt/squashfs /squashfs
mount -o bind / /mnt/squashfs

mksquashfs /mnt/squashfs /squashfs/filesystem.squashfs -comp gzip -no-exports -xattrs -noappend -no-recovery -e /mnt/squashfs/squashfs/filesystem.squashfs
find /boot -name 'vmlinuz-*' -type f -exec cp {} /squashfs/vmlinuz \;
find /boot -name 'init*' -type f -exec cp {} /squashfs/initrd.img \;

and in packer download artifacts from vm:

    {
      "type": "file",
      "direction": "download",
      "sources": [
        "/squashfs/vmlinuz"
      ],
      "destination": "output/{{user `name`}}-squashfs/{{user `name`}}.vmlinuz"
    },
    {
      "type": "file",
      "direction": "download",
      "sources": [
        "/squashfs/initrd.img"
      ],
      "destination": "output/{{user `name`}}-squashfs/{{user `name`}}.initrd"
    },
    {
      "type": "file",
      "direction": "download",
      "sources": [
        "/squashfs/filesystem.squashfs"
      ],
      "destination": "output/{{user `name`}}-squashfs/{{user `name`}}.squashfs"
    }

[adam-erickson]

Sorry, but can anybody explain these answers more clearly for a n00b? I get the general idea and particularly like the Packer and squashfs solution of @vtolstov, but some of the details are unclear. This is ripe for a blog post.

[vtolstov]

if you combine you question, we can answer it. bare metal is very specific, somebody needs pxe boot, somebody need installer. May be some of users uses rpm-ostree.... =))

[adespain]

What do you do with the artifacts? I'm also a confused n00b on how to implement this so the hardware will boot with the packer image...

[eduncan911]

@adespain PXE boot means you setup your DHCP server to answer queries on your network. This usually involves setting up a TFTP server to host the "artifacts" you build with packer per the processes above.

You configure the remote machine for PXE boot in the BIOS, which will boot and use PXE to ask the DHCP server for an IP and usually includes the PXE instructions to pull an image from and boot it.

DNSmasq is a good bit easier to configure for this since it does both dhcp and dns, and allows for PXE configuration. You only need to setup an tftp server and be done.

Should be plenty of articles out there on how to setup on PXE boot and where to put the images, aka artifacts.

---

A few tips: configure multiple images and configure your DHCP server to answer the query based on the MAC address of the machine. When a specific MAC address comes in, point to a specific image you have packer'd and host in a specific sub-directory on the tftp. This was my plan as I have four I want to run k8s cluster as, and all others have specific IoT-around-the-house functions.

It's a pita to find each Mac of each machine you want to PXE boot (I have like 12 raspberry pis I wanted to setup). And not all machines support PXE boot But it's a one-time thing.

The alternative is to have only 1 image and answer all PXE boot queries with the same image - which may or may not work for your setup.

I haven't had the time to built it yet myself (building a house right now!). But the above has been per a lot of research and duckduckgo searches. :) I personally have a bunch of UP Board devices as they all support PXE and run full Intel kernels. RPi3 does but I haven't tested it.

[adespain]

Is there a way to not use pxe boot and just install the packer created image from a usb drive or something?

[rubu]

I followed the excellent points from @vtolstov here, but as many others I needed to make an ISO for UEFI HTTP boot. I followed the instructions from https://willhaley.com/blog/custom-debian-live-environment/, to make an ISO but it fails to boot (I formulated a question on Server Fault with all the steps - https://serverfault.com/questions/999778/booting-a-squashed-filesystem-from-iso).

Can anyone explain to me as person which is not that familiar with grub and linux boot process, what am I missing here (since I obviously feel like I am :D). Do I need something in the boot chain (dracut?) that will do smth to load the squashfs and launch the actual OS or is this just a grub configuration issue?

[rubu]

So, going through all the stuff on Google, I actually moved a lot forward (leaving this here so that int future anyone who stumbles on this can get through:D):

So in the provision script I added:

dracut -a dmsquash-live -N -m "kernel-modules base" --filesystems "squashfs" /dracut/initrd.dracut.img

And I put this initrd (not the one from the original system) in the ISO and use dracut parameters root=live in the grub boot entry:

menuentry "Custom Linux" {
    linux /vmlinuz root=live:/dev/sr0 rd.live.debug=1
    initrd /initrd.img
}

So this at least seems to boot the squash fs, but now I get

Warning: /dev/mapper/live-rw does not exist

so back to Google

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.