Scrcpy: Non english keyboard layout

Created on 11 Mar 2018 · 38Comments · Source: Genymobile/scrcpy

I have hebrew keyboard layout on both my computer and my android keyboard that is in use but it wouldn't let me pass the letters. Any chance to fix this?

feature request input events keyboard unicode

Source

elig0n

👍22

Most helpful comment

One possible technical solution is to write a customized input method manager. Vysor(an scrcpy alternative) does this.
The input method manager can be installed and enabled by adb, without user interaction.

wuairc on 12 Mar 2018

👍5

All 38 comments

Any change to fix this?

Probably not. Only ASCII and some accented characters work.

Except if someone finds a technical solution ;)

rom1v on 11 Mar 2018

I used to have the same issue but it disappeared through some "practice":

Turn to AOSP and type some numbers like 1234 (that works fine).
Type some letters. If it doesn't work, press shift key several times. (You may say a bunch of letters "crowding in".
Turn to GBoard or other keyboards. It will work fine if you are lucky :)

MiracleXYZ on 11 Mar 2018

wuairc on 12 Mar 2018

👍5

+1 on this. Wanted to write in Hebrew too, but failed.
Are you sure it's impossible to use adb to pass unicode keys ?

AndroidDeveloperLB on 7 Apr 2018

@rom1v One of the possible options could be to create a custom keyboard, like NULL keyboard that I described here: https://github.com/Genymobile/scrcpy/issues/181, but with an ability to Input and Copy/Paste unicode. Probably not the easiest option, but at least something if there's no system API for such things.
AFAIK, KDEConnect installs own keyboard.

vshymanskyy on 22 Jun 2018

I would prefer without this workaround...

AndroidDeveloperLB on 23 Jun 2018

I strongly hope this will be fixed soon

giorgiobeggiora on 21 Sep 2018

https://remoboard.app/
example of this app may be help.
pass unicode via an wireless server independent from adb server.

livin2 on 26 Oct 2019

@livin2 This means I have to change my default keyboard, which means I might have less features than what I want to have.
If an app is required, I would even install one that requires root, but having less features and having to switch keyboards as an alternative... This is just a hassle...

Is there maybe a way to do it using "monkeyrunner "? Someone wrote about this here:
https://stackoverflow.com/a/20615725/878126

AndroidDeveloperLB on 26 Oct 2019

If an app is required, I would even install one that requires root, but having less features and having to switch keyboards as an alternative... This is just a hassle...

feature like emoji mostly depend on the ime of computer.
and you can switch keyboards with adb shell in the script where you start scrcpy:

adb shell ime list -s

adb shell ime set 【IME NAME】
adb connect 【xxx】
scrcpy -s 【xxx】

livin2 on 26 Oct 2019

Yes, still a hassle. Once I disconnect from USB, I have to switch back to the keyboard I use.
Besides, all this time that I use Scrcpy I won't have any of the features of the keyboard that I use.

AndroidDeveloperLB on 26 Oct 2019

Refer to the Vysor method, add a custom ime to enter unicode characters

903

pangliang on 8 Nov 2019

I guess it may not be that easy and other apps may implement it successfully.

hope can input and copy/paste unicode characters via scrcpy window directly.

NewUserHa on 7 Feb 2020

hope can input and copy/paste unicode characters via scrcpy window directly.

Yes, this works. https://github.com/Genymobile/scrcpy#copy-paste

rom1v on 7 Feb 2020

I mean 'unicode directly'.

what about putting the key event to the IME if don't implement something new?

NewUserHa on 7 Feb 2020

I think if users are playing mobile game using scrcpy, they must need type texts sometimes.
if the users are trying to type/paste chinese or japanese then scrcpy is unable to cover that needs. and I don't think scrcpy need other big features except this one.

NewUserHa on 7 Feb 2020

I don't think scrcpy need other big features except this one.

and #14 :wink:

rom1v on 7 Feb 2020

oh, I'm just reading it while you reply.
the audio seems a bigger feature and some hard and a little important than current one. but when waking up the phone(cpu855) using scrcpy, I feel it recover from lag in the first seconds(on windows). I guess there could be problems if there's audio playing in the background.

NewUserHa on 7 Feb 2020

Could copy-paste work method work for non-English?
I tried now using it, but it didn't paste.
It did succeed to copy it to the clipboard of the device though.
Is there maybe an Android-supported key for pasting ? If so, maybe this could be a workaround for typing...

AndroidDeveloperLB on 7 Feb 2020

When you Ctrl+Shift+v in the scrcpy window, the computer clipboard is copied to the device clipboard (exactly, without problems with special characters).

You can then long-press in any text area on your fevice and click in Copy.

rom1v on 7 Feb 2020

@rom1v I know. That's why I wrote "It did succeed to copy it to the clipboard of the device though.".
My question was if it's possible to do a real pasting, perhaps using some workaround of paste via a key.

AndroidDeveloperLB on 8 Feb 2020

Please test #1426 (works on Android >= 7).

rom1v on 25 May 2020

any workarounds found?

JulyMorning on 25 Jun 2020

POC that I implemented before.

https://github.com/npes87184/SocketIME

And its demo.

npes87184 on 27 Aug 2020

@npes87184 So this is by using a new keyboard app on the device, that has to be set as default, right?

AndroidDeveloperLB on 27 Aug 2020

@npes87184 To keep things simple on the client side, I think we could do the following:

put the IME service class in scrcpy-server
if the user pass --use-ime (for example), then the scrcpy-server installs itself (programatically, it has shell permissions) asynchronously (if the same version is not already installed), without delaying video stream and control (except text events, which could be queued or ignored)
it then enables the IME and connects to it
once connected, all text events received by scrcpy-server are now forwarded to the IME
on cleanup, scrcpy-server restore the previous IME

Tht way, the client always communicate to scrcpy-server only, which manages everything. And this doesn't delay scrcpy starting (except for text).

What do you think?

rom1v on 27 Aug 2020

👍1

@npes87184 So this is by using a new keyboard app on the device, that has to be set as default, right?

In this POC, yes.

@npes87184 To keep things simple on the client side, I think we could do the following:

put the IME service class in scrcpy-server

if the user pass --use-ime (for example), then the scrcpy-server installs itself (programatically, it has shell permissions) asynchronously (if the same version is not already installed), without delaying video stream and control (except text events, which could be queued or ignored)

it then enables the IME and connects to it

once connected, all text events received by scrcpy-server are now forwarded to the IME

on cleanup, scrcpy-server restore the previous IME

Tht way, the client always communicate to scrcpy-server only, which manages everything. And this doesn't delay scrcpy starting (except for text).

What do you think?

I would say that this method is pretty well.
However, there is a problem about UI. How do we "make" an area to let user enter words.

npes87184 on 27 Aug 2020

However, there is a problem about UI. How do we "make" an area to let user enter words.

Oh, I've never used this kind of things. I don't really know how they work.

Isn't this area provided by the system, and give the resulting character as a "text event"?

Once we have the text event with UTF-8 content (typically for one "character", like for letters), we can inject it to the system using the IME without being limited to ASCII.

rom1v on 27 Aug 2020

and what about the physical keyboard? making user type on the device directly.
and what about adding a text control let user input texts and copy to the device by scrcpy in bachground when user type enter.
both ways will be better than panful installing anything.

NewUserHa on 27 Aug 2020

Is there perhaps a way to achieve unicode key support by using accessibilityService?
Maybe using dispatchKey() or something?
Seems to me like a better thing, instead of having to use new keyboard and reset it...

AndroidDeveloperLB on 27 Aug 2020

However, there is a problem about UI. How do we "make" an area to let user enter words.

Oh, I've never used this kind of things. I don't really know how they work.

Isn't this area provided by the system, and give the resulting character as a "text event"?

Once we have the text event with UTF-8 content (typically for one "character", like for letters), we can inject it to the system using the IME without being limited to ASCII.

Oh, you misunderstand my question. My question is how do we design an area in SDL so that user can type in that area and send it to IME.

npes87184 on 27 Aug 2020

https://wiki.libsdl.org/SDL_StartTextInput
https://wiki.libsdl.org/Tutorials/TextInput
(not tested)

rom1v on 28 Aug 2020

@rom1v What's that?

AndroidDeveloperLB on 28 Aug 2020

当手机打开投屏时，系统会自动添加wfd-uhid-device的物理键盘
这让scrcpy的input keyevent可以被输入法监听到
这很神奇，但是我不知道如何在不开启投屏的时候启用这个wfd-uhid-device物理键盘
视频测试经过地址：
https://raw.githubusercontent.com/vip-liutao/scrcpy/main/keyboard.mkv

vip-liutao on 20 Oct 2020

当手机打开投屏时，系统会自动添加wfd-uhid-device的物理键盘
这让scrcpy的输入键事件可以被输入法监听到
这很神奇，但是我不知道如何在不开启投屏的时候启用这个wfd -uhid-device物理键盘
视频测试通过地址：https : //raw.githubusercontent.com/vip-liutao/scrcpy/main/keyboard.mkv

@rom1v

vip-liutao on 20 Oct 2020

How do you enable "wfd-uhid-device" in practice?

Refs:

https://github.com/Genymobile/scrcpy/issues/87#issuecomment-378775565
https://github.com/Genymobile/scrcpy/issues/279#issuecomment-453819354

rom1v on 20 Oct 2020

When I activate raw key events mode (by pressing Ctrl+K) then try to type using Arabic (101) keyboard layout, I get messages like these on command line and no Arabic letters are passed to my Android device: