Dietpi: General | Update VM images to Stretch

[Fourdee]

• 🈯️ VMware
• 🈯️ VirtualBox Many thanks @MichaIng !

[MichaIng]

Manual upgrading VirtualBox image to Stretch worked without any error by the way.

At least for testing purpose a QEMU image would be nice as well, as it provides "real" emulation of ARM processors and even natively provides a RPi2 machine. I tried to get the RPi2 DietPi image running on it, but it starts with boot loops (same as the default Raspbian Lite image). Something needs to be adjusted with the SD partitions, then it works, but the DietPi first start scripts break it again.

[Fourdee]

• 🈯️ VMware is done: http://dietpi.com/downloads/images/DietPi_VmWare-x86_64-(Stretch).7z

[MichaIng]

@Fourdee
I could help to build the VirtualBox image. How do you start? Using the official Debian CD image and manually go through system preparation and finalize scripts?
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/

Also if I see the preparation script: Couldn't this be automated also by defining some hardware+distro variables at the top and run the ??? parts accordingly? Or is there some other reason you run the commands manually, better control/validation possibilities, or other issues?

[Fourdee]

@MichaIng

Or is there some other reason you run the commands manually, better control/validation possibilities

Exactly this. If we did automate it, we'd need to check each command output for errors.

I've considered automating the installing as you mentioned with HW_MODEL etc, but we'd need to improve fstab generation in both prep and finalize scripts, that populates current mounts, then puts in our additional entries.
Possibly a few more things that put me off automating this, however, cant remember at the moment 😄

I could help to build the VirtualBox image. How do you start? Using the official Debian CD image and manually go through system preparation and finalize scripts?
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/

By all means go for it 👍

• Debian 9 net install iso
• 8-20GB VM disk, expanding (not preallocated/fixed size)
• 1 partition (EXT4), no swap
• when prompted to install software (eg: desktop/ssh/tools), make sure all are unselected
• Once its ready, start system prep.

Although, it may be easier to simply convert the VMware disk to VirtualBox? 😄

[MichaIng]

@Fourdee
Okay, will do. Another question: Is there a reason why some packges are purged first and reinstalled afterwards, e.g. clean config files or something?
I want to use the chance to do another image afterwards, testing things mentioned here: https://github.com/Fourdee/DietPi/issues/1146
for comparison and testing.

Although, it may be easier to simply convert the VMware disk to VirtualBox? :smile:

No idea, is the setup/hardware-imulation related stuff the same between VMware and VirtualBox? Did you try this in the past? Would be another thing to test and compare otherwise :smile:.

[k-plan]

@MichaIng
By all means go for it :+1:

Yes, please.

@Fourdee
Okay, will do.

Thank you very much.

Did you try this in the past?

Some user fiddling around with image converting virtual disk here: https://github.com/Fourdee/DietPi/issues/804

http://www.sytone.com/2017/07/getting-a-dietpi-image-running-on-hyper-v/

They used qemu-img for Windows.

But never test it.

[MichaIng]

Some user fiddling around with image converting virtual disk here

Interesting. Could save time of course, on the other hand we do not create images thaat often :wink:. But I am still thinking if the emulated hardware and thus dietpi steps are the same. On the other hand, all the config adjustment is anyway done on first run setup. If I find the time, I will create the image both ways and compare them.

[Fourdee]

@MichaIng

Another question: Is there a reason why some packges are purged first and reinstalled afterwards, e.g. clean config files or something?

I was going to say "I forgot to remove them from removal" however,:

• Purging clears all configuration files, ensures we get clean configs to work with once reinstalled.

So no loss and maybe some gain 😄

[MichaIng]

@Fourdee
Hehe learning by doing: Some questions/remarks about installation:

Debian first run asks quite a lot of questions:

• All localize things (language, keyboard, WLAN,...) set to GB / en_GB.UTF_8
• Domain name? Should not matter yet, right? Just chose localhost.
• Enabled shadow passwords, seems good.
• Just kept root user with password "dietpi".
• Single partition with expending size up to 12 GB (Jessie image was 8, but Stretch simply takes more space by default), it is quite challenging to get around the installer automatically creating swap partition^^. What to choose on creating new partition table? Don't find ext4 there. I used guided partitioning, removed both partitions afterwards (including swap) and creating a single new one in whole free space.
• Chose firmware-linux-amd64 kernel, which will be updated to current version via apt.
• No additional tools, no virtualbox guest extension and no generic firmware installation also? I also skipped the USB firmware installation, as the installer finds the in VirtualBox preconfigured virtual USB hub. Deactivated this also now by switching virtual pointer to PS/2, which is anyway not used in console.
• There was a question about efi boot loader and to move it to some removable media part or something in case of buggy images. I just skipped that, bootup afterwards worked fine.

Now to the preparation script:

• How do you practically go through the commands? As VirtualBox does not support copy&paste on console, I first installed dropbear and used ssh for copy&paste.
  _€: Just found the optional openssh installation. Changed this to dropbear in new PR, as this anyway will be installed later, or was there a special reason for openssh?_
• By default apt tries to access to IPv6, for my system I needed to add Acquire::ForceIPv4 "true"; to apt settings. Will remove before image creation. But as IPv4 works on every system, we could use this as default and let users choose explicitly to enable/force IPv6? Same for wget on some servers, but this scripts here worked fine.
• On purging/installing the packages I got:

dropbear: WARNING: Invalid authorized_keys file, remote unlocking of cryptroot via SSH won't work!

I guess I can ignore, since I don't use key authentication here?

• In between I am asked to choose if/which keymaps to install, I choose to not touch it/leaf it as is.
• echo -e "CONF_SWAPSIZE=0" > /etc/dphys-swapfile should be btw after package installation, as installation of swapfile asks again to overwrite that config.
• Chose MBR: grub2 asks where to install to, I chose just the whole disk (sda) instead if the partition (sda1).
• WLAN Firmware is not necessary? I installed it now, maybe someone wants to use WLAN USB stick for some reason?

• /boot/dietpi/dietpi-logclear: line 243: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  Info:

  • Log file directory | /var/log
  • Processed files | 22
  • Excluded files | 0
  • Cleared log files | 0
  • Unsupported files | 0
  • Deleted files | 22
  • Space cleared | 15213 KB

  We could symlink /DietPi/dietpi to /boot/dietpi first? But as it's just the notification...

root@dietpi:~# mount -a
mount: special device /dev/mmcblk0p1 does not exist

fstab expects sd card, thus manual adjustment of fstab necessary.

• Copy&Paste cats does not fill empty lines. Have to take care about this, but think I added all empty lines manually.
  _€: Copying from edit file page includes spaces!_
• My ethernet interface up by start was called enp0s3, thus /DietPi/dietpi/func/obtain_network_details didn't work and I needed to follow https://kernelpanik.net/rename-a-linux-network-interface-without-udev/ to rename it to eth0.

During finalize script somehow my internet connection broke. So wget could not download something. Needed to adjust interfaces for my environment and somehow needed to set eth0 to auto (instead of allow-hotplug) before it worked.

I didn't add much meta data to the appliance: http://dietpi.com + v158.

Finally I ended up with bootloops. First I thought it is because I forget to add a boot.ini. Chose the one from Odroid C1, which I also found on my Jessie VM image. But afterwards same result:
Log shows:

Failed to start Remount Root and Kernel File Systems
See systemctl status systemd-remount-fs.service for details.
...
Failed to start Create Volatile Files and Directories.
See systemctl status systemd-tmpdiles-setup.service for details
...
[Failed] RootFS is currently set to R/0.
...

Finally I resolved it by opening the image in another VM, find the UUID by ls -l /dev/disk/by-uuid and added it to boot.ini and fstab. Interesting is that UUID in fstab and boot.ini of the old Jessie VM does not match. Seems the one from fstab is actually not used?

I sent you the link to the image. Somehow it got >400 MB, no idea why. Will check again tomorrow and also do another, where all packages but the wanted ones are marked as auto installed (instead of purging the selection) and autoremove will do the rest. Just to compare.

[MichaIng]

Okay, leaned some things:

• x86 does not use /boot/config.txt nor boot.ini nor cmdline.ini
• To boot up correctly one just need to change /etc/fstab to correct UUID. Perhaps it would be easier to just replace this with /dev/sda1 for VM images.
• Before running finalise, I need to change (after interface name change to eth0) eth0 interface to auto. As allow-hotplug one of the earlier commands in finalise script breaks connection and wget commands afterwards fail. interfaces file will be reset during finalise anyway.
• I created a new image now. Seems that every access to disk via other VM increases image size and it can't be reduced anymore to initial size by running finalise script afterwards.
• I left firmware-linux-nonfree and wifi firmware packages away now. Hardly anyone will use wifi nor will simulate any special hardware through VirtualBox. Otherwise firmware can be installed easily.
• I know there is some setting in dietpi.txt, but I would force apt IPv4 by default: Acquire::ForceIPv4 "true"; Every system will work with this and e.g. we could also disable IPv6 by default and by enabling it (if someone knows he uses IPv6 and configures his subnet accordingly) also force IPv4 can be removed again. I would do the same for curl and wget, but I didn't found a way to force it there. Adding -4 as option to curl/wget works, perhaps we can create some hook command and remove that again after ipv6 is enabled again?
  _€: Add, for wget it's echo "prefer-family = IPv4" >> /etc/wgetrc_

New image uploaded to link in email. No idea how to make it smaller than 310M, wonder how the VMware Stretch image can be that much smaller 🤔.

[Fourdee]

@MichaIng

New image uploaded to link in email. No idea how to make it smaller than 310M, wonder how the VMware Stretch image can be that much smaller

After finalize:
I boot into a Linux env (eg: Hirens boot cd), then zerofree the disk, sets all free space to 0, resulting in much smaller compressed file.
zerofree -v /dev/sda1

http://www.hirensbootcd.org/download/

[MichaIng]

Used just another VirtualBox instance to manipulate the image.
Worked like a charm! New archive uploading, 180M instead of 318M 😃.

[Fourdee]

@MichaIng

Just tried the VirtualBox image you created, works a charm, great work and many thanks 👍

All uploaded and switched in http://dietpi.com

[Fourdee]

Completed, thanks again @MichaIng for creating the VirtualBox image 👍