Qmk_firmware: The Apple Fn Key

[fauxpark]

_I've made this an issue instead of a PR because it probably warrants some discussion on whether or not to add this feature into QMK, but either way I think it's a good idea to at least put it in the documentation in the event that it's not implemented, so that anyone sufficiently determined can at least get some pointers on how to do so on their own. Anyway..._

I wanted to see if I could get QMK to send the infamous "Apple Fn" key. It turns out I can, and it's fairly simple.

First of all, I discovered this interesting Xcode project called HID Explorer, which details all of the usage pages and usages of all your HID devices, and shows their current values. You may want to be somewhat familiar with Xcode and Objective-C to use it, though -- I had to do a little fiddling to get it to compile. Someone should really fork this and patch it up!

If you look at an Apple keyboard in HID Explorer, you can see that right at the bottom of the list is a strange entry: page 0xFF, usage 0x03. This just so happens to correspond to the usage page AppleVendorTopCase and the usage KeyboardFn, which Apple has kindly documented for us. Sure enough, pressing the Fn key causes the value of this usage to become 1.

You can also see the keyboard is indeed 5KRO as has been mentioned in previous discussions on this key. The reason for this is because the keyboard report has an extra field for the Fn key, which _replaces_ the sixth keycode:

|Byte|0 |1 |2 |3 |4 |5 |6 |7 |
|----|----|--------|--|--|--|--|--|--|
| |Mods|Reserved|1 |2 |3 |4 |5 |Fn|

But it doesn't _have_ to take up a keycode slot. Because the report descriptor tells the host how to interpret the report, we can just repurpose the reserved byte for the Fn key field and it will work perfectly fine, at least within macOS. I don't know why Apple didn't just do this considering it doesn't have to deal with old BIOSes.

So, I set about replacing the declaration of the reserved byte with the Fn key descriptor elements. Then, I created a new action, ACT_APPLE_FN, and a new keycode KC_APPLE_FN and glued it all together, with an additional check in add_key_to_report() and del_key_from_report() so that they modify keyboard_report->reserved instead. In my keymap I added KC_APFN and KC_F3, compiled & flashed, and wouldn't you know it, I can toggle Exposé!

Here is the diff - it's only a proof of concept, but IMO it's pretty minimal.

But there's a catch. And it's a big one. The keyboard's vendor and product ID need to match those of a real Apple keyboard -- probably only ones with a Fn key. The product ID also seems to determine whether certain F-keys work eg. Launchpad/Mission Control and keyboard backlighting. The Manufacturer and Product strings can still be whatever you like, but this explains why there are no third-party keyboards with Apple Fn keys, and most likely rules out QMK compatible PCB/keyboard sellers from shipping boards with the key available out of the box too. I can imagine that being a bit of a dealbreaker.

Lastly, something funny I discovered: with my WASD V2 in Mac mode, it shows up in HID Explorer with the Fn key usage. When you press 6 keys at once, the value of this "Fn key" becomes the keycode of the sixth key! It looks as if WASD simply copied and pasted the report descriptor from a real Apple keyboard, but kept the same report format. So you still only get 5KRO in Mac mode, and the Fn key also does not trigger, because the vendor and product IDs are not Apple's.

[skullydazed]

This is very cool, but I don't know how we possibly include this. Leaving this open to discus the possibility for now.

[drashna]

@skullydazed Short of maybe a feature option that overrides the normal 6KRO code. (eg "APPLE_NKRO", maybe).

But not sure how Apple would feel about us doing that, given that we have to use their VID and PID to get this working.

[fauxpark]

There would have to be some kind of warning if the feature is enabled, reminding users that it won't work unless they change their VID/PID - putting those values into the repo, even for checking, is probably a bad idea. I can see this may not be very "QMK".

I've also found that the PID can determine functionality amongst Apple keyboards - on my 2011 MBP, the F4 key has a little gauge symbol on it and opens the Dashboard. On my 2017 MBP it has a different symbol, and opens Launchpad.

QMK is meant to be customised though, even if you can't push your cool "Apple" keyboard back to the repo, we should at least be able to say that it is technically possible.

[jsoltren]

I noticed that matias keyboards offer a Mac version with an fn key. So I picked up a matias wired aluminum keyboard, plugged it into my Mac, and checked out what happened.

This is really interesting:

`Matias Wired Keyboard:

Product ID: 0x024f
Vendor ID: 0x05ac (Apple Inc.)
Version: 1.91
Serial Number: FK318VUS
Speed: Up to 12 Mb/sec
Manufacturer: Matias Keyboard
Location ID: 0x14210000 / 16
Current Available (mA): 500
Current Required (mA): 100
Extra Operating Current (mA): 0`

Looks like they just straight up lie about their Vendor ID. 0x024f is not a known Apple product ID, in that it doesn't appear in pci.ids:

https://usb-ids.gowdy.us/read/UD/05ac
http://www.linux-usb.org/usb.ids

[Syzygies]

Karabiner Elements can remap arbitrary keys to mission_control, launchpad, or other Mac uses for function keys.

It is good engineering to choose the right layer in which to work. I'm constantly revising VSCode-specific key mappings as I work, by editing a CSV file that updates Karabiner via Goku, and an on-screen crib sheet. One wants one's keyboard firmware to be fully functional in case a software layer like Karabiner isn't present, but it makes no sense to be reflashing a keyboard's firmware all the time. Rather, you're here to design a keyboard you can live with. So involving Karabiner is a good thing. For example, it can implement tap or hold without a timer, so the key fires on key-up no matter what the delay, if it isn't used as a modifier. I haven't figured out how to do this yet in QMK.

I use the Mac function key internally in Karabiner, as a modifier flag for internal use that I can be sure no application is expecting. I won't be using that trick with QMK. However, the Mac also equivalences left and right modifiers, so no app can be looking for both left control and right control, for example. This is another way that QMK can set modifiers to be trapped by Karabiner, without interfering with any downstream app. Karabiner sees the left, right distinctions, but apps don't.

[jsoltren]

Karabiner Elements does its remapping with a kernel driver. It's the "right" way, inasmuch as having to hack the kernel to work around the USB stack is considered right.

Apple is constantly trying to clamp down on kernel development. Karabiner Elements was itself a rewrite of Karabiner. In that regard QMK is more "stable" across OSes.

[Syzygies]

I'm likely to contribute to QMK once I get my sea legs. Yes, there was a lapse without Karabiner, and Apple is warning again of impending changes to kexts.

The "Oryx" key on an ErgoDox EZ or Planck EZ sets up two-way communication with QMK firmware. It would be ideal to continually tweak key macros on the fly, through a program that helped the user as the Oryx trainer does, without having to either reflash the keyboard or rely on kexts. The Proton C, for example, certainly has the memory and processing power. I can live without mission_control, and I'd love not to be worried about the future of kexts.

[Nuru]

It's April 2020 and things seem to have changed. I have a 2019 MacBook Pro with a Microsoft full-size ANSI keyboard:

  Product ID:   0x00db
  Vendor ID:    0x045e  (Microsoft Corporation)
  Version:  1.73

and it has the "Insert" key mapped to the "fn" key under macOS 10.15 Catalina. Even the Mac Keyboard Viewer shows the insert key as labeled "fn" (under El Capitan, the keycap was labeled "?⃝" for "Help"). I also note that the "fn" key on the built-in keyboard sends a keycode just like every other key; it does not seem to be special in any way.

So I would hope that QMK can now map a key to "fn", or failing that, can map a key in such a way that Karabiner can map it to "fn". At the moment, the Dygma Raise (using QMK) can do neither, and is not going to try, citing this QMK issue as the reason. It would be great to have this feature at least workable if not fully supported.

[skullydazed]

@Nuru That's an interesting observation. Do you have the microsoft software installed for your keyboard?

[jsoltren]

@Nuru that's really interesting. I wonder if this is related to particular keyboards, particular versions of the macOS, particular systems (can't rule out macOS behaving differently on your 2019 system), or if this indicates kernel level change in the macOS. I did not see this behavior on earlier versions of macOS 10.15.

I have a Mid-2012 MacBook Air running 10.15.4. I definitely don't see this behavior with my Novatouch.

Could it be specific to the VID/PID mentioned earlier? It would be a good experiment to flash a QMK keyboard with this VID/PID to see if we can reproduce this result.

[Nuru]

@skullydazed No, I do not have any special software installed for the keyboard.
I realize now that I do have Karabiner-Elements 12.9.0 installed, and that is handling the mapping of "Insert" to "fn" somehow. I don't know if/how it is changing the keycap in the keyboard viewer. I did spend a lot of time in the past trying to get the "fn" key on the Dygma Raise to work as an "fn" key, without success, so I may have set this up, but it seems unlike me to choose the "Insert" key as the key to map to "fn". I would tend to choose a completely unused key like "Scroll Lock" or "Print Screen" rather than "Insert", which still is used by some applications.

@jsoltren I have an 2013 MacBook running 10.14 Mojave and the "fn" key sends a keydown and keyup event. It is not just some internal modifier.

Also, I just got my 34 key keypad

  Product ID:   0x2012
  Vendor ID:    0x05c7  (Qtronix Corp)
  Version:  1.00

and Karabiner maps its "Insert" key to "fn" no problem.

It's possible a lot of this is being done by the Karabiner kernel extension. I have quit the application and seen the mapping go away, but the keycap stays and the built-in keyboard's function key stays the same. I have not tried fully uninstalling Karabiner yet.

The other interesting thing is that Karabiner-EventViewer reports the "fn" keycode as 65538. That is outside the range of a 16 bit number. It is binary 1 0000 0000 0000 0010. This suggest the kernel extension is at play.

My general attitude about all this is that Karabiner is a thing, lots of people use Macs, and it is not unreasonable to ask QMK to implement a feature that only works in conjunction with Karabiner. It would be great if QMK could get the "fn" key to work without Karabiner, but if that is too hard, it would still be helpful for QMK to support a special keycode that you then feed into Karabiner as "fn" so that I don't have to give up an existing key for it.

[jsoltren]

@Nuru Karabiner, being implemented with an actual kernel extension, enables emulation of the true fn key at the kernel level. I use this myself on some systems and I've glanced at the Karabiner sources.

@skullydazed may have a different opinion but my view is that the firmware should not be doing anything special for some particular kernel or user program. You can program any key to send any key code, and catch that in kernel or user space for your need. This seems sufficient to me.

What I do support, is an optional build flag for faking an Apple VID/PID. Apple wouldn't be too happy about this shipping, but what you do on your own keyboard is between you and your console.

@Nuru, if you have a chance, could you try completely uninstalling Karabiner, restarting, and trying again? There are instructions for a full Karabiner uninstall (which is necessary in this case) on pqrs.org.

[Nuru]

At the very least, I can say that one thing that has changed since this issue was created is that it is no longer true that "there are no third-party keyboards with Apple Fn keys".

@jsoltren I did a full uninstall of Karabiner and rebooted, and nothing changed that didn't change when I quit Karabiner.

I recall now that what got me started down this path was seeing Apple's external Magic Keyboard with Numeric Keypad, which I thought was a normal USB keyboard, has the "fn" key where ANSI keyboards have the "Insert" key. So I thought any USB keyboard could do it.

Now I realize that (1) it is a Bluetooth keyboard, not USB, and (2) Apple provides special treatment for their own peripherals. It does, at least, explain how "fn" ended up on the "Insert" key on the Keyboard viewer and why I tried that mapping in Karabiner.

On the other hand, I note that the Logitech Wireless Solar Keyboard K750 (Mac version) also has the "fn" key in the same place (replacing the "Insert" key), so it is not strictly limited to Apple keyboards, and although that keyboard is wireless, it wirelessly connects to its own USB dongle, so it connects to the computer via USB, not Bluetooth. And it says it requires no software installation, so it is not using its own kernel extension.

I don't know how to sniff the USB connection between the Logitech Keyboard and the computer to see what its "fn" key is generating on the bus, but I'm hoping someone else can sniff that and see about duplicating it in QMK. The Logitech keyboard is sold by Apple, so it is certainly possible it has special support of some kind, but if you need to go down the route of faking a Vendor ID, it might be safer to fake Logitech than to fake Apple.

[fauxpark]

Wireshark on macOS is able to sniff USB packets, though from Mojave or Catalina onwards you need to disable SIP for the interfaces to show up (and then you have to ifconfig <blah> up them).

[uchuugaka]

Were these merged in by a PR at all?
I was able to use the changes in https://gist.github.com/fauxpark/010dcf5d6377c3a71ac98ce37414c6c4
in my own local and it worked like a charm to turn a keyboard into having a proper Fn key as macOS keyboards use them and as a knock on effect makes the F keys work as media keys without extra effort when Fn is used (or reverse depending on OS settings).
The changes were well enough #ifdef isolated and pretty clear. The diff itself also ends up being a quick and dirty tour of where things really happen in QMK and how. Almost a nice intro tutorial. (providing the consumer of the tutorial has a mac)

[fauxpark]

@uchuugaka unfortunately due to the legal implications of imitating an Apple device (by setting their VID and PID), this probably won't be able to be merged. Unless you have managed to get it working using some other VID, in which case that is quite interesting.

[Nuru]

@fauxpark @uchuugaka FWIW I have the Fn key working on my Mac running Catalina with a Microsoft keyboard, Vendor ID 1118, PID 219. Not that imitating Microsoft is legally better than imitating Apple, but just to say that I does not have to be an Apple device to work.

[fauxpark]

Maybe things have changed since High Sierra. I think that was when I last properly tested this. I'll try it again in the morning.

[uchuugaka]

I'd be surprised if it were an issue since Matias has been using the vendor ID for decades.
The product ID does not have any effect in my observations.
If the Microsoft vendor ID also works, that implies there are some IDs that work as known to Apple (or more likely, they're formally registered vendor IDs with usb.org or with somehow registered with Apple.
https://www.usb.org/getting-vendor-id
As far as I know it is quite literally just a formality that donates money to usb.org (USB-IF) and earns the rights to using official USB logos and stuff.
(it also gives you a 16 bit integer's worth of product IDs with usb.org https://hackaday.com/2015/04/03/usb-pids-for-all/ )

Effectively, it's like registering a port number like 666 the port for Doom for network play. Nobody can or will really chase it because it is 100% spoofable like a web browser's User Agent string or Caller ID.

The USB standard also handles this because each device gets a unique ID assigned automagically by the bus itself and no OS in their right mind would predicate anything secure against VID or PID.

FWIW, Apple even share it in constants…
https://developer.apple.com/documentation/usbdriverkit/3295284-apple_s_vendor_id/kiousbapplevendorid
https://developer.apple.com/documentation/usbdriverkit/usb_device_descriptors
https://opensource.apple.com/source/IOUSBFamily/IOUSBFamily-396.4.7/IOUSBFamily/Headers/USBSpec.h.auto.html

If it helps, the driver is 100% Apple's HID device driver unless you provide something via their DriverKit.
All USB keyboards are the same except what scan codes they produce unless you provide additional software via DriverKit.
(like Karabiner does and an interposer)
By default Apple's USB HID stack loads drivers that match the device type.

[fauxpark]

I'm aware of all that - my suspicion is more that Apple maintains a list of "allowed" VIDs that the Fn key will work on. It feels very in keeping with them to do something like that. Obviously it's not meant to be "secure", just a first-level deterrent against impersonating Apple.

RE Matias, it seems as though they have/had some kind of relationship with Apple, if this (admittedly quite old) press release is anything to go by. So it is possible they have been granted permission to use Apple's VID, or Apple has whitelisted Matias devices somehow.