Qmk_firmware: `OSL(layer)` doesn't seem to work as expected

[zosiu]

So I'm trying to create a dead-key layout, and from the readme, OSL seemed like way to do it.
I set up the following keymap: https://gist.github.com/tomb0y/5a3e815ca8d7efb8e23d0e2dd635d0e0

The comma key on the first layer is set to OSL(2).
Based on my the readme, I'd expect the following:

• while on the first layer, press ,
• the second layer is activated, no charcode sent
• press let's say #, then that charcode is sent out, and the active layer goes back to the first
• so pressing let's say a after all this should send a and not #

But it seems like I'm stuck in the second layer after pressing ,.

Am I misunderstanding how OSL should work, or is this a bug?

[algernon]

For OSL to work properly, the same key on the target layer should be KC_TRNS. That's because OSL activates on tap, and sets itself up to be cleared on keyup (this is so that you can hold an OSL key and have the layer active for the duration). In your case, on keyup, you're on a different layer, and a different key is in the OSL key's position, so it fires the keyup for that.

You can also work this around by turning . into a macro that sends the keycode, and clears the one-shot state on keyup. Something like this should work:

case OSL_DOT:
  if (record->event.pressed) {
    register_code (KC_DOT);
  } else {
    clear_oneshot_layer_state(_DK);
    unregister_code (KC_DOT);
  }
  break;

[zosiu]

Works great @algernon, thanks a lot!

[zosiu]

@algernon one more question: I'd still want the dead key to act as a regular comma if pressed with shift (producing <), is there anything I can do about that?

[algernon]

That's going to be a bit more involved, as you can't use the OSL macro directly then. Turn it into a macro, which - for the sake of the example below - I will call M_DKCOMM. This macro would check if we have a keydown event, and if so, if shift is pressed too, register a comma, otherwise switch layers and set it up for one-shotness. On keyup, it will clear the one-shot layer state, and unregister the comma unconditionally.

The code for this looks something like this:

case M_DKCOMM:
  if (record->event.pressed) {
    if (keyboard_report->mods & MOD_BIT(KC_LSFT) || 
        keyboard_report->mods & MOD_BIT(KC_RSFT)) {
      register_code(KC_COMM);
    } else {
      layer_on(_DK);
      set_oneshot_layer(_DK, ONESHOT_START);
    }
  } else {
    clear_oneshot_layer_state(ONESHOT_PRESSED);
    unregister_code(KC_COMM);
  }
  break;

The problem with this is that when tapped once, on keyup, this macro is not called, but KC_2 is released. You'll need to turn that into a macro too...

In this case, the above one can be simplified, you can drop the clear_oneshot_layer_state part, and create another macro, replacing KC_2 in the _DK layer, with this:

case M_2:
  if (record->event.pressed) {
    register_code(KC_2);
  } else {
    clear_oneshot_layer_state(ONESHOT_PRESSED);
    unregister_code(KC_2);
  }
  break;

I have not tried if this works, but it should. The thing to keep in mind when it comes to layers is that if you switch layers when a key is pressed, when it goes up, it will fire an up event for the key in the switched-to layer. So if you want to do something on keyup too, you have to do that in the other layer.

Hope this helps!

[zosiu]

Many thanks for the quick and clear response @algernon , works like a charm!
I have a fully working dead-key layout now thanks to you 👍

(I will send in a PR after tidying up the code a bit, maybe it will be helpful for others too.)