K3os: Arm ISO is not bootable for Raspberry Pi 3

[miry]

I download arm64 iso and burn in MacOS via Etcher.

I got next partitions on SD card:

/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *15.9 GB    disk2
   1:       Microsoft Basic Data                         71.7 KB    disk2s1
   2:                        EFI NO NAME                 2.9 MB     disk2s2
   3:       Microsoft Basic Data                         384.1 MB   disk2s3

Should it be MBR boot loader?

[miry]

Here is example of Fedora image:

/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *15.9 GB    disk2
   1:             Windows_FAT_32 NO NAME                 79.7 MB    disk2s1
   2:                      Linux                         512.8 MB   disk2s2
   3:                      Linux                         2.5 GB     disk2s3

[kisskillkiss]

Same Problem.
On my Sopine A64(arm64) don't install K3OS.

[Plaquevent]

Hello, I am also very interested in running K3OS on Raspberry Pi 3B+ too. Could you please provide an image or explain how to get it working ? I cannot get to boot on the ISO file (whether it is burnt on an USB device plugged to the RPi or on it's SD card). Would the bootstrapped, takeover or ARM overlay installation work ?

[fransvanberckel]

The same counting, if you dd the ISO to USB. The USB doesn't boot on AMD64 bare metal.

[zimme]

I've got it running on raspberry pi's using Ubuntu server for Raspberry pi and doing an Arm overlay installation of k3os.

https://ubuntu.com/download/iot/raspberry-pi-2-3
https://github.com/rancher/k3os#arm-overlay-installation

[debianmaster]

same issue on mac. downloaded iso, converted to img , flashed to sdcard and pi is not booting.

[stossy]

Same issue as OP

[Tchoupinax]

Hello,

Same issue. My tests about remastering iso are sucessful on a VM. But when i tried with the arm iso and on my raspberry, it does not boot.

[dmc5179]

If the partition table is to be believed I suspect the issue is that the k3os boot partition is EFI:

Number  Start   End     Size    File system  Name                Flags
 1      32.8kB  104kB   71.7kB               Gap0                hidden
 2      104kB   3054kB  2949kB               EFI boot partition  boot, hidden
 3      3054kB  416MB   413MB                Gap1                hidden

AFAIK there is no EFI solution for the Raspberry Pi

I was able to get K3Os to install using the ARM overlay but once booted it doesn't appear to have everything running.

[debianmaster]

+1 for this

[dweomer]

The arm64 ISO is currently targeted at server-grade hardware. The overlay method is currently the only way to get installations of k3OS on arm and arm64 working on Raspberry Pi 3/4.

[miry]

@dweomer Do you know, if there are any plans to automate build ISO/IMG for Raspberry Pi 3?

[dmc5179]

@dweomer I've tried the ISO overlay several times starting with Ubuntu on Pi 3 and then overlay. Each time I find that most services are not running when I reboot. If I start K3OS AMD64 in a VM it starts correctly every time. I'm not sure the overlay method works for Pi3.

K3OS ARM 64, the lightweight kube for the edge, is targeted for server grade hardware.... that sounds weird....

[zimme]

@dmc5179 have you properly updated the file /boot/cmdline.txt by adding rw? Otherwise things won't work

[dmc5179]

@zimme yes, I did add rw to the cmdline.txt file. If I don't do that and reboot the OS won't boot at all.

[dmc5179]

I even went so far as to try to boot a qemu arm VM on x86_64, which you can do. My plan was to boot to K3OS ARM VM on x86_64, attach SD card to VM and tell K3OS in the ARM VM to install to what would be the SD card. Then see if I can boot the SD card on a Pi. I can boot a fedora ARM VM but it requires the kernel and initramfs. I haven't found a way to get qemu to boot ARM ISO.

[dweomer]

@dweomer I've tried the ISO overlay several times starting with Ubuntu on Pi 3 and then overlay. Each time I find that most services are not running when I reboot. If I start K3OS AMD64 in a VM it starts correctly every time. I'm not sure the overlay method works for Pi3.

There is no "ISO overlay". For RPI 3 boards, as per the ARM Overlay Installation instructions in the README, you will need to unpack the appropriate rootfs tarball on top of a working root filesystem installed by the likes of Raspbian or Ubuntu Server for RPI. If you use Raspbian as a base you should know that it is 32-bit (because it targets RPI 2/3) and therefore the arm64 overlay will not work (because you don't get the k3OS kernel with ARM overlay, you get the kernel from your base install). If you use Ubuntu for RPI 3 that @zimme linked to above then you do have a 64-bit installation and you should use the arm64 tarball.

A decent introductory tutorial: https://blog.alexellis.io/test-drive-k3s-on-raspberry-pi/#preparetherpi

[dweomer]

@dweomer Do you know, if there are any plans to automate build ISO/IMG for Raspberry Pi 3?

There are no plans for this currently but that doesn't mean won't happen.

[Ognian]

The arm64 ISO is currently targeted at server-grade hardware. The overlay method is currently the only way to get installations of k3OS on arm and arm64 working on Raspberry Pi 3/4.

@dweomer I just did the following:

1. follow Installation in https://github.com/pbatard/RPi3 -> this results in a UEFI partition on a otherwise empty sd card
2. copy k3os-arm64.iso on a USBB stick
3. connect sd card, usb stick, monitor and keyboard to raspberry pi 3 (should work with 3+ too and soon with 4) and power up

After choosing to start k3OS LiveCD & Installer from the grub menu k3os boots on the Raspi.
I started os-config but did not let it do its job since I don't want the UEFI partition to be overwritten...

If I manually partition the sd card and lable a partition K3OS_STATE, will then the installer pick this one and not overwrite the whole sd card?

Ognian

[sjdevries]

@Ognian , Have you had a chance to try the above and see if the ISO actually works with UEFI?

@dweomer , would having an installer option for force MBR partitioning allow SOC's to use the arm image? Or are there other driver/firmware issues that are resolved by the arm overlay installation, that would also require a separate pi3/4 ISO to be generated?

[dweomer]

If I manually partition the sd card and lable a partition K3OS_STATE, will then the installer pick this one and not overwrite the whole sd card?

@Ognian specifying the --no-format flag combined with the K3OS_STATE partition device, e.g. /dev/mmcblk0p2 (assuming uefi is on /dev/mmcblk0p1) should prevent the overwrite of the uefi partition

[dweomer]

@sjdevries wrote:

@dweomer , would having an installer option for force MBR partitioning allow SOC's to use the arm image? Or are there other driver/firmware issues that are resolved by the arm overlay installation, that would also require a separate pi3/4 ISO to be generated?

I haven't looked into arm/arm64 packaging at all as we have yet to declare official support for either arch. I would like to get it done but it is low on my list of priorities for the time being.

That said, if your or anyone in the community would like to contribute a PR that makes arm/arm64 bootable media (with the caveat that for arm64 we would prefer to not break the AWS packer build) I would be thrilled to review it. 😃

[fransvanberckel]

Okay, check. If someone wants to investigate. It looks the build is done with a Docker container. And the Dockerfile is calling grub-mkrescue and xorriso = wrapper ...
https://github.com/rancher/k3os/tree/master/images/06-iso