DietPi-RAMdisk | Replace with vmtouch to use native fs caching

[MichaIng]

I am not 100% happy with DietPi-RAMdisk. Changes to those files should not rely on a clean shutdown, which is currently required to write RAMdisk back to disk. As well it causes confusion when files are present at two locations.

Actually since Linux has a very good disk caching system already, why not using that one. It caches often used files/scripts automatically. But if we really want DietPi scripts to be most responsive even when being called the first time, I found some tool which is able to lock files in cache: https://github.com/hoytech/vmtouch

Available from Debian repo since Buster: https://packages.debian.org/buster/vmtouch
But I would compile own binaries and implement with systemd unit, since the sysvinit service is broken anyway: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=947603

Tested it and works as expected. Executing locked example scripts from /boot/dietpi results in exactly same read speed compared to tmpfs. The vmtouch binary itself is extremely lightweight, the overall used memory matches nearly exactly the locked files, hence not more then a few KiB overhead, most likely similar compared to an active tmpfs.

Requires some more testing but actually not much risk. Backwards compatibility can be simply achieved via symlink: rmdir /DietPi; ln -s /boot /DietPi

One single current issue: /DietPi/dietpi/.network needs to be global write-able currently to allow non-root users updating current active network adapter + IP (e.g. dietpi-banner) as well. But such flag files, which shall not survive boots, are better placed in /tmp or /run anyway.

---

Argument summary:

• Our scripts cannot break if DietPi-RAMdisk fails to mount or move scripts from disk to tmpfs.
• Changes are not lost if DietPi-RAMdisk fails to move files back to disk on shutdown, e.g. in case of power loss.
• Users are not confused that changes to /boot/config.txt|boot.ini are not boot-persistent. Even that those files contain explaining comments, we have regular reports.
• Speeds up boot process since vmtouch is much faster then DietPi-RAMdisk service + script.
• vmtouch is then completely optional, speeding up our script executions and settings reads, allowing flexibel additional cache locks or releases. If it's removed or fails, disk caching falls back to Linux, which automatically caches often accessed files anyway.
• From disk writes perspective: Settings changes within our scripts then cause direct disk writes, merged according to async. However especially in combination with async those are minimal compared with other tasks, e.g. simple "apt update" causes magnitude of order larger writes. And more importantly with DietPi-RAMdisk, all DietPi scripts and settings are completely written to disk on every shutdown (and dietpi-update), which very likely is much more compared to edited settings files in a session.

[Trunkzeh]

@MichaIng Are you going to push this as part of a 6.27.3? I'm going to be moving the majority of my internal services off my RPI0 to the RPI4 once 6.27 is stable; which will require a clean reinstall. The RPI0 will then be repurposed for beta testing so will be happy to test things that way.

[MichaIng]

@Trunkzeh
Nope, definitely not something for v6.27, as I aim to release before new year and this change is too deep to do required testing and all the code changes in this short time. I just wanted to write down the idea to not forget about this 😉.

I'll keep you updated, would be great if you want to test it, when ready.

[Joulinar]

@MichaIng
you know we are always happy to test new stuff. At least my 2 cents to support these great project

[MichaIng]

I compiled vmtouch packages: https://dietpi.com/downloads/binaries/buster/
They are basically compatible with the default Debian packages, hence if they compile a newer version, it will continue to use the existing config file etc. I used a version string suffix "-dietpi1" that is treated as newer than all Debian patch version suffixes, but older if a new upstream version string is present.

By default the shipped config file locks /boot/dietpi and /boot/dietpi.txt in file system cache, however this can be changed via config file /etc/default/vmtouch, or the service can be disabled (systemctl disable --now vmtouch) or uninstalled (apt purge vmtouch) without breaking anything.

---

Things can be tested with this branch: https://github.com/MichaIng/DietPi/pull/3402

G_DEV_BRANCH ramdisk

Works so far on my tests, as well done on older Jessie and oldest stable images to verify update path is not broken.
Only thing I want to add is to create stat and flag files (time sync exit status, update available flag, network info etc.) inside /run/dietpi/, so they are never written to disk and clearly separated from settings/config files which we want on disk to be boot persistent of course.

[Joulinar]

ok did a quick test on my RPi4 and so far update to v6.29 was working without issues. As well next reboot was working fine. No error messages on journalctl.

[MichaIng]

@Joulinar
Many thanks ❤️, so far so good 👍.