Pmbootstrap: Small Onscreen-Keyboard for initramfs to enter LUKS password

[lawl]

We need a small (as in binary size and dependencies) on-screen keyboard to enter the LUKS password to decrypt the root partition.

Potential starting point to fork off: https://github.com/zrafa/onscreenkeyboard

Other pointers to code/tools we could use welcome.

[ollieparanoid]

Some recommendations from research I made a while ago:

• When writing a new keyboard from scratch:
  
  
  
  • Try to use an established keyboard layout format (something from GNU/Linux or TWRP/AOSP?)
  
  
  • Try to support framebuffer/dri/x11/wayland if possible. Pay attention to DRI at least, because that is what should be used in favor of framebuffer in mainline Linux.
  
  
  • it might make sense to statically link it against SDL2, so you'd get all these platforms and it shouldn't be that big
  
  
• It may make sense to have the keyboard in ladscape-view in the initramfs
• There is an existing GNU/Linux on screen keyboard standard with dbus. Certainly nothing for the initramfs, but if you intend your keyboard to be usable outside of the initramfs, you might be able to dynamically load dbus .so library and integrate into the desktop, if you want?

With that being said, even the simplest keyboard, that just works, would be a huge improvement compared to the telnet typing we do now. Keep it simple :)

[MartijnBraam]

It's also important to watch what gets started in initramfs since that is not in a crypto partition but running processes stay loaded after unlocking, If someone flashes a new boot partition without the phone owner knowledge it could insert a rootkit in the initrd that has access to the unlocked rootfs. I think we shouldn't share too much between the initrd and the main system

[ollieparanoid]

This is a bit off-topic, but I think when an attacker is able to flash a new boot partition without the owner's knowledge, then it does not matter what was or was not on the boot partition.
I'm all in for stopping what is running in the initramfs before entering the normal init though.

We can discuss this more on IRC, if you guys want to, but let's keep this out of the ticket.

[ollieparanoid]

Thanks to @MartijnBraam's efforts, we have a device with a keyboard working now. It would be nice, if the unlocking screen also supported typing in text with a real keyboard (but if it can't be done easily, then ignore this - we could also disable showing the on-screen-keyboard unlocking method, when the keyboard is set to true in the deviceinfo).

[MartijnBraam]

The on-screen keyboard should be hidden on the n900 but this tool should probably also take over the console show the input box to enter the password in, otherwise the kernel log wil draw through the keyboard

[lawl]

Sorry, busy with IRL stuff for a few days. I will be back.

[ollieparanoid]

No problem @lawl, thanks for telling us that.

locusf on Matrix suggested using cuteboot. (Personally I think QT is rather heavy for the initramfs, but I have not looked into how cuteboot strips it down etc.).

locusf: yes its basically a minimal bootstrap of qt for initramfs
locusf: theres maliit too
me: do you happen to know if it would have a on screen keyboard out of the box?
locusf: it does not afaik

[lawl]

Personally I think QT is rather heavy for the initramfs, but I have not looked into how cuteboot strips it down etc.

I would agree, from the top of my head pmos ramdisk+kernel is already ~10mb, while I think the default android one is for my maguro ~5mb. Repartitioning sucks and I don't think we can do LVM shenanigans in boot.

I'll take a look at that likely tomorrow. I compiled the onscreenkeyboard I linked and tried it on maguro today. Basically it draws the (very small) onscreen-keyboard. And I hooked up the mouse version (osk_mouse.c) to the touchscreen instead of the mouse. Swiping in any direction on the touch-screen did move the selection forward.

So from what I see so far, the changes that would have to be done to osk_mouse.c are:

• Draw in landscape mode, not portrait.

  • This is easy, it's just a PPM that's drawn on the framebuffer, we can just prerotate that.

• Read the actual location of touch events from /dev/input/x

  • No idea, but should be doable I think.

Things I don't have an answer to yet:

• We don't want to carry around multiple PPMs for different resolutions to keep the ramdisk small.

  • Have the device package depend on the correct resolution ppm? Urgh

• We don't want to carry around multiple PPMs for different keymaps for different keyboard layouts (QWERTY, QWERTZ...)

  • Maybe the solution here is to drop keyboard layouts and only provide QWERTY, with only lowercase alpha characters, maybe numbers? Shift, region specific characters (eg. german umlauts) and other special characters are a pain in the ass to type anyways on an onscreen keyboard. So I do believe the best things to use are passphrases anyways.

[ollieparanoid]

Amazing progress!

Just a quick note about space running out: We could also make a dedicated partition inside the system partition (which is partitioned anyway in pmOS)/sdcard, that holds unencrypted data for the initramfs.
We could put a file on the already unencrypted boot partition, and mount it in the initramfs. That would mean, that the user had to flash the system partition every time the initramfs changes, though (or update the on screen keyboard package, which then writes to /boot).
So if that becomes a concern, I think we could handle it - but it would be better if we wouldn't need to.

One keyboard layout for now is good enough I think, we should just add a message before creating the password in pmbootstrap, that the user shouldn't use special characters for now.

About the ppms... if this is the easiest way, we could carry around different ppms for different resolutions for now. maybe vector graphics in the long run - or drawing with a font to the screen?

[MartijnBraam]

If we add the screen resolution + dpi to the device info then the keyboard package can generate the required PPM's in the packaging step

Or supply the keyboard images in the device package itself (automated or manual).

[ollieparanoid]

Adding the screen resolution + dpi to the deviceinfo sounds like a good idea, this could become useful in more cases later. I'd prefer that over duplicating the ppm files in device-packages.

[MartijnBraam]

Using yamui from sailfish might also work. From the readme:

Small application to draw progress bar, logo, text or short animations to the
screen. Suitable for minimal environments where there are no other frambuffer
users. E.g. in initrd or early boot process splash screens.

https://github.com/sailfishos/yamui

[ollieparanoid]

Removed the future label. We've made decent progess, and this could really come in handy now :+1: Does not look like @lawl is working on this anymore, so feel free to pick this up dear reader.

[ollieparanoid]

@drebrez split the initramfs, which gives us more available space. This will be useful for the on screen keyboard! See also: Initramfs development

[ollieparanoid]

@craftyguy and @MartijnBraam have started a SDL-based implementation here:
https://github.com/postmarketOS/osk-sdl