Mattermost-server: Keyboard layout issues with shortcuts

[ArchangeGabriel]

Summary

Keyboard shortcuts seem to ignore the actual layout.
For instance, when typing Ctrl+V, I get the File Uploading window, because my V happens to be where the U is on Qwerty layout notably.
Note that this is happening in Chromium, Desktop app (well that’s a bit like Chromium), and Firefox with a default profile but not with an hardened one (which I guess prevent you to access the actual keycode instead of the resulting input).
Also, when actually typing Ctrl+U I don’t get the File Upload, but the default browser shortcut matching this i.e. source code.

Steps to reproduce

Use a keymap where for instance the V is at the position of a Mattermost defined shortcut (U, K…). Try to Ctrl+V something inside the text box, or try to Ctrl+U.

Expected behavior

Content should be pasted if you do Ctrl+V, file upload should open when doing Ctrl+U.

Observed behavior (that appears unintentional)

Other actions are triggered instead.

Possible fixes

It seems that not being able to access the underlying key code is solving the issue, so I guess don’t try to do clever things?

[rocketnova]

Also confirming that this is happening for me. I typically use Dvorak, but after the most recent upgrade cmd+k using the k in dvorak (v in qwerty) is not working. However, cmd+k using the k in qwerty works no matter what input source I'm using, including when I'm typing everything else in dvorak.

[jasonblais]

Thanks for the report! @ArchangeGabriel @rocketnova this should be resolved in v5.2 via https://github.com/mattermost/mattermost-webapp/pull/1515

[rocketnova]

@jasonblais Great! Will that resolve the issue in the desktop app, too? That's where I've been experiencing the issue.

[jasonblais]

@rocketnova It should - our QA team will test on both browser and desktop app though, so if not we'll catch it.

[rocketnova]

@jasonblais Awesome. Thank you!

[amyblais]

I'll close this Issue for now as v5.2 has been released - feel free to reopen this if the issue persists after upgrading.

[ArchangeGabriel]

Sorry, but it still does not work with 5.2.

[ArchangeGabriel]

(And I apparently cannot re-open)

[ArchangeGabriel]

Let’s be more precise: there is some progress. Ctrl+V still gives the File Upload instead of pasting content, but Ctrl+U now correctly gives the File Upload.

[amyblais]

Thank you @ArchangeGabriel, I'll do testing and create a ticket for this soon.

[amyblais]

@ArchangeGabriel What OS are you using? Our QAs weren't able to repro this on Mac.

[ArchangeGabriel]

Linux.

[amyblais]

What version?

[ArchangeGabriel]

Arch, latest Chromium/Xorg/any other involved component.

[aeris]

Same trouble here.

Using bepo layout leads to CTRL+V binds to "upload a file" like CTRL+U (which is the physical equivalent of V on a AZERTY keyboard).
CTRL+U works correctly, like CTRL+C and CTRL+X.

Seems a regression, works correctly on 4.8.0, but not on 5.1.1 nor 5.2.1.
Same behavior on the desktop (4.1.2-2 on arch) and web clients.

[amyblais]

Update: Our QAs are working on testing this and I'll let you know their findings as soon as I get their feedback.

[amyblais]

We haven't been able to repro this internally - would you like to do some testing on our nightly build server and see what you experience there?

[aeris]

Still the trouble on nightly-build :cry:

[amyblais]

Hi @ArchangeGabriel @aeris - would you be able to share any logs / Console errors you see from around the time the issue occurs in case those would give an indication on the cause of the issue?

[aeris]

There is no error or log. Seems to just open the built-in file selector when pasting on the text input field.

[aeris]

Regression seems between 5.0 and 5.1.0.
Docker image mattermost-preview is OK in 5.0 but not 5.1.0.

[aeris]

5.0.3 ok too, but not 5.1.0-rc1
No other available docker version between those releases.

[aeris]

OK, after git bisecting, seems this is the culprit commit : https://github.com/mattermost/mattermost-webapp/commit/4d27296f10c2e0f5d5680f18eeee47c4297a5668

[aeris]

Commenting out this part restore the paste shortcut.

Here is a debug output of the CTRL+V sequence on my layout of the isKeyPressed

[amyblais]

Thanks all! I created a ticket for this here for our developers as I believe there is enough information to investigate this: https://mattermost.atlassian.net/browse/MM-11926.

[amyblais]

@ArchangeGabriel @aeris Hey all, the ticket (https://mattermost.atlassian.net/browse/MM-11926) was discussed today with the team and were wondering if one of you would like to help with the fix for this? Our team might not be able to reliably fix / test this issue as we don't normally have access to Dvorak and Bepo keyboard layouts.

If you're not sure how to fix it, I can also open the ticket as "Help Wanted" for all community members to look at!

[aeris]

Yep, no problem to try to fix this :smiley:
But for testing, you don't need a physical dvorak or bepo keyboard, just change your keyboard layout on your favorite window manager :blush:

At debugging, I show some strange "key" value associated to keyboard event.
You can notice in the screenshot above that the 3 CTRL and "v" events are associated to "k", then "u", then "/" keys. Is it expected ?

[amyblais]

@sudheerDev

Pinging you as you've fixed other keyboard layout related issues and I wasn't sure who is most familiar with this:

The reporters are seeing an issue where Ctrl+V doesn't work properly on Dvorak and Bepo keyboard layouts - instead of being able to paste text, they get the File Upload window.

They are able to help fix this but have a question: in the screenshot above the 3 CTRL and "v" events are associated to "k", then "u", then "/" keys. Do you know if this is expected? If you're not sure, I can ask other devs for input as well.

[sudheerDev]

@amyblais I am familiar with these issues I will take it from here.

@aeris
Shortcuts we support are of two types.

1. For English keyboards with different layouts (Qwerty, Dvorak etc)
2. Different language keyboards(Russian etc)

So, we support two types of events from keyboard.

1. Physical key position i.e for layouts like Russian though there is no equivalent of cntrl+k we use physical position to check i.e though event.code
2. Actual key for layouts like qwerty, dvorak we use event.key or event.keyCode.

For users with Russian keyboard л is mapped to k so people can trigger quick switcher.
Same case with 'v' mapping to 'k'. so it fine as it works for keyboards like dvorak

I don't seem to replicate these issues with dvorak/other language layouts.

I don't have bepo keyboard layout on mac. Is that supported by linux by default or are you running a keymap for simulating bepo?.

[aeris]

OK, i see. Complicated :smile:
Bepo is natively supported on Linux.
Driver for Mac could be find here.

[amyblais]

The Jira ticket has been resolved and tested internally - the fix will be available in v5.4 release,

[marcimat]

Does someone know when Mattermost Desktop Application 5.4+ will be available in download for macOS ? I use Mattermost Version 4.1.2 (4.1.2.46) and I have this embarrasing behavior in fact (Cmd+V opens file upload box in Bépo keyboard) !
The https://mattermost.com/download/ page does not propose it yet.

(and thanks for the fix by the way !)

[amyblais]

Hi @marcimat, v5.4 is the current Server version and v4.1.2 is the current Desktop app version. Can you help confirm what Server version you're currently on? You can find this information at Main Menu > About Mattermost.

You can download the current Server version (v5.4) near the top of the download page https://mattermost.com/download/.

[marcimat]

Oh! Sorry, I think I misunderstand (I saw that there is a 5.4 version in mattermost/webapp, but the last mattermost/desktop is 4.1.2 indeed). So. I use the last Desktop (4.1.2.46) on my computer.

The Server version we use (embedded with Gitlab) is "Mattermost Team Edition Version: 5.2.2" ; we need to upgrade the Server version so ? I thought it was a problem with the Desktop App.

Thanks for your answer.

[ArchangeGabriel]

I can confirm that this is indeed fixed with 5.4 release! Congrats on that. :)

[MayaWolf]

This still/again happens to me on the 5.9.1 version. I'm using a non-QWERTY keyboard layout with QWERTY key codes, and it's incorrectly using key texts rather than key codes for keyboard shortcuts. Please reopen this issue.

[jasonblais]

@MayaWolf Thank you for letting us know. Would you be open to sharing the following to help us try reproduce the issue:

• operating system this reproduces on, eg. Windows 10, Mac 10.12, ..
• browser, eg. Chrome, Firefox,

/cc @amyblais

[MayaWolf]

@jasonblais Thanks for getting back to me!
Windows 10, Chrome.
My keyboard layout is similar to Dverty (Dvorak key texts, Qwerty key codes; the purpose being that you can keep using Qwerty keyboard shortcuts while typing in Dvorak)
The most likely reason for this issue is that key texts are used instead of key codes for processing keyboard shortcuts.

[amyblais]

Hi @MayaWolf, would you be open to testing if this reproduces on the latest version? You can test on our community server which is on v5.11.

[MayaWolf]

Yep @amyblais I just tried it. I try to press Ctrl+V to paste, and it brings up the "Type to find a channel" dialog (because it looks at the key text K instead of the key code V)

[sudheerDev]

@MayaWolf i checked this. I don't have Dverty but i checked this with Dvorak with qwerty and it works as expected.

I wonder if it is related to your keyboard layout. What does cntrl+t in browser for you?

[MayaWolf]

If I press Ctrl+T (i.e. the key that would be T on a Qwerty keyboard and is Y for me), I get a new tab. Because Chrome (and most other programs) read key codes, not texts, for keyboard shortcuts.
A key event for Ctrl+V looks like this for me: (filtered to show only relevant properties)

altKey: false
charCode: 0
code: "KeyV"
ctrlKey: true
key: "k"
keyCode: 86
metaKey: false
shiftKey: false
type: "keydown"
which: 86

I assume that you're reading the key property, which returns the text for typing, not the key code.

[ArchangeGabriel]

Well, there is no solution that can work for everyone here. If you use keycode, which is likely what was used before, it will break for normal Dvorak and Bépo users, because they do use Qwerty keycode underneath.

Under a normal browser, when I do a (Ctrl+)V, I would get a U keycode (under my hardened Firefox I don’t because it blocks JS from knowing the real keycode).

[MayaWolf]

Dvorak keyboards by default use Dvorak key codes as well. You have to specially set it up to not do that. That's the entire point of key codes - for things that are about the raw keys (like keyboard shortcuts and in games) you use the code, and for things that are about the _text_ of keys (typing) you use the key label/text. Dvorak keyboards and keyboard layouts by default send key codes that match Dvorak, unless you set it up so they don't.

In other words, if I wasn't using a specially modified version of the Dvorak keyboard layout, a keypress of Ctrl+V (what would be V on a Qwerty keyboard) would look like this:

altKey: false
charCode: 0
code: "KeyV"
ctrlKey: true
key: "k"
keyCode: 75
metaKey: false
shiftKey: false
type: "keydown"
which: 75

I tested this with the Dvorak keyboard layout included in Windows.
The code property still returns the key on a Qwerty keyboard (because that's the point of the property), but keyCode matches the K key.

[ArchangeGabriel]

OK, honestly I don’t know about Dvorak, never used it. But I can definitively tell you I didn’t do anything special to get Qwerty keycode while typing with my keyboard, and that I have never seen any application (excepted Mattermost before) react on keycode instead of keytext.

Also this might be a difference between Windows (you) and Linux (me).

[ArchangeGabriel]

From a fast look up online, it seems that on Linux at least, keycodes are indeed always US Qwerty ones, whatever the actual layout.

[MayaWolf]

You're correct; it may very well be a difference between Windows and Linux in that regard. On Windows, literally every application (with very few exceptions which are very annoying when you're using a non-Qwerty keyboard layout) will use key codes for shortcuts.

Is there any way to change that behaviour on Linux? I would say the problem here is not how Mattermost was handling keyboard shortcuts, but that your keyboard setup sends keycodes that mismatch your key texts. I feel like that's not something that should happen by default. A keyboard layout is supposed to emulate a different keyboard after all, not just change the text for typing (and because it was only doing the latter is exactly why you were getting mismatched keyboard shortcuts in Mattermost).

[ArchangeGabriel]

I don’t know. But unless you want to advocate for this change Linux-wide, it does not really makes sense. However it means that Mattermost should likely use keycode on Windows, but not on Linux.

[jasonblais]

@sudheerDev any thought on the latest notes above?

[sudheerDev]

@MayaWolf
https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/keyCode

keyCode is deprecated and it does have cause issues last time i checked. We do fallback to keyCode if there no key match as of now. Though there is no good JS property that has good browser coverage as of now.

Even popular keybinding libraries are flooded with issues with layout and language keyboard issues.

There is no universal solution to this yet so we did try to make hot keys work with as many different layouts as possible which include not just different type of english keyboards but also language keyboards.

@jasonblais We can have an another look at our implementation but IMO there is no universal solution to this.

[jasonblais]

@sudheerDev Do we know if there's a standard (or recommended) approach taken by other products, and if we are following such approach?

[sudheerDev]

Do we know if there's a standard (or recommended) approach taken by other products, and if we are following such approach?

@jasonblais I would say we do but most products don't support as many keyboard layouts and as many shortcuts as we do which causes issue like this IMO

Even a propular library as https://github.com/ccampbell/mousetrap/issues still tons of issues around layouts and language keyboards.

[MayaWolf]

Honestly...maybe make a per-user option for it? I know, I hate the idea too, but right now for me I can't even copy-paste anything in Mattermost without using the right-click menu, and if there really is no way to achieve cross-platform compatibility otherwise...? Or maybe a platform-specific (depending on how well it can be detected) branch on which event property to use. I agree that there is currently no good cross-platform solution for this in the JS KeyboardEvent API, which is a real shame.

[jasonblais]

Hey all, had a conversation with our team - currently it appears that Mattermost keyboard shortcuts support standard languages and keyboard layouts, but they may not work if you use keymapping that overwrites default browser shortcuts.

If you would like to see support for a per-user option or a wider support for keymaps, would you be open to contribute this in the feature idea forum so it can be discussed, upvoted and considered for a help wanted ticket?

Please include a link back to this GitHub Issue. If you're interested in implementing, please say so and we'll prioritize the review.

You get 10 votes in the feature idea forum, and each one influences the future of the project.

[amyblais]

Closing as this can be followed in the feature request forum,