Scrcpy: Support touchscreen input

[flipflop97]

My laptop has a touch screen but trying to control my phone with it doesn't seem to do anything at all. It would be nice if touch input was also supported.

[rom1v]

Hmm, I thought a touch screen just generated mouse events.

Does it work with a mouse?

[parkerlreed]

Works with a mouse. Touchscreen is just flat out ignored. Acts as a mouse elsewhere which is weird. It still shows the cursor going over the scrcpy window but tapping/dragging does nothing.

[MCMic]

I guess you need to catch https://wiki.libsdl.org/SDL_TouchFingerEvent
https://wiki.libsdl.org/SDL_MouseMotionEvent#Remarks does say that mouse motion can have a touch input as source so it seems consistent with the fact that moving works and not clicking.

[renedecandido]

I had the same issue on Linux and an external touch monitor. As a workaround you can use Touchégg with some configuration and additional gesture
<gesture type="TAP" fingers="1" direction=""> <action type="MOUSE_CLICK">BUTTON=1</action> </gesture>

[liaoqingmo]

just mouse event

[parkerlreed]

Any update on this?

[rom1v]

Hi,

I started to implement it, but I have no touch screen, so I cannot test.

If someone want to test (and help to fix if it does not work), I pushed a branch: finger.

[parkerlreed]

It seems to be way too sensitive towards long presses. And I don't think it's matching the location where I'm touching.

[rom1v]

@parkerlreed Thank you for your feedbacks. I'm not very surprised :smile: I really need to get a laptop with a touchscreen.

[rom1v]

I got a computer with a touch screen! (thanks @bonrry!).

Here is my current wip: finger.5.

It works in image viewers or OSMAnd, but not in webbrowsers (I can't zoom with multitouch in Firefox, while it works on the real device). I don't know why yet.

[rom1v]

Seems to work now: finger.5.

Please give me your feedbacks. I will rewrite the commits properly when I have time.

[rom1v]

Here are some binaries, if you can't build it:

• a release for Windows: scrcpy-win64-v1.10-31-g783eb95.zip (_SHA256: a286f7b4532e0b0a29fd758ba1f32826079eb621b7a2194cbfa3b02e3aa3b517_)
• the prebuilt server: scrcpy-server-v1.10-31-g783eb95.jar (_SHA256: 942878b2eaa93930441865b0520ef1217aba7f8dbfb1a152e4713ec7a093f730_)

[parkerlreed]

Figured I'd post here too for good keeping

[rom1v]

Merged in dev branch.

[unclehowell]

Quick question, if I update my files to reflect the changes you've made on the development branch can I expect touchscreen to work, or will I have to rebuild from source/ wait for the snap release? ??

[rom1v]

Quick question, if I update my files to reflect the changes

Which files?

You can reflect the changes on the source files, then build from the (modified) sources, but then why not just take the dev branch and build?

[unclehowell]

I replicated all the changes to all the files you made on Oct 02 & 3rd. Rebooted and reloaded everything. Touchscreen inputs still not recognised by scrcpy. I tried manual build from source before but didn't work. Ended up using snap

[rom1v]

What did you do exactly? What did you "reloaded"? If you edit the source files, you need to build from these new sources…

[unclehowell]

I noticed the last update to the Snap package was the 5th October so I tried re-installing with Snap using the edge channel. No touchscreen. I removed snap and I am now revisiting building from source (a git clone of the ´dev´ branch), but building from source has issues, which I´m looking into. To be more specific I get these errors after running the command: meson x --buildtype release --strip -Db_lto=true

WARNING: Project targetting '>= 0.37' but tried to use feature introduced in '0.48.0': console arg in custom_target DEPRECATION: build_always is deprecated. Combine build_by_default and build_always_stale instead Build targets in project: 8
WARNING: Project specifies a minimum meson_version '>= 0.37' but uses features which were added in newer versions:
* 0.48.0: {'console arg in custom_target'} Found ninja-1.8.2 at /usr/bin/ninja

If I then switch to a non-root user and run the ninja command, like the guide suggests, I get the following error message: \ninja: error: opening build log: Permission denied. So I try sudo ninja and get the following error: [36/39] Generating scrcpy-server with a custom command. (not invoking gradle, since we are root) [39/39] Linking target app/scrcpy.

If I proceed to do sudo ninja install I get this error [0/2] Generating scrcpy-server with a custom command. (not invoking gradle, since we are root) [1/2] Installing files. Installing app/scrcpy to /usr/local/bin Stripping target 'app/scrcpy' using strip. Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/mesonbuild/mesonmain.py", line 129, in run return options.run_func(options) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 521, in run installer.do_install(datafilename) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 343, in do_install self.install_targets(d) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 436, in install_targets raise RuntimeError('File {!r} could not be found'.format(t.fname)) RuntimeError: File 'server/scrcpy-server.jar' could not be found FAILED: meson-install /usr/local/bin/meson install --no-rebuild ninja: build stopped: subcommand failed.

I also ran release.sh while playing around and this is what I got:

The Meson build system
Version: 0.52.0
Source dir: /opt/scrcpy
Build dir: /opt/scrcpy/build_test
Build type: native build
Project name: scrcpy
Project version: 1.10
C compiler for the host machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
C linker for the host machine: GNU ld.bfd 2.30
Host machine cpu family: x86
Host machine cpu: i686
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency libavformat found: YES 57.83.100
Run-time dependency libavcodec found: YES 57.107.100
Run-time dependency libavutil found: YES 55.78.100
Run-time dependency sdl2 found: YES 2.0.8
Configuring config.h using configuration
Program ./scripts/build-wrapper.sh found: YES (/opt/scrcpy/server/./scripts/build-wrapper.sh)
WARNING: Project targetting '>= 0.37' but tried to use feature introduced in '0.48.0': console arg in custom_target
DEPRECATION: build_always is deprecated. Combine build_by_default and build_always_stale instead.
Build targets in project: 8
WARNING: Project specifies a minimum meson_version '>= 0.37' but uses features which were added in newer versions:

• 0.48.0: {'console arg in custom_target'}
  Found ninja-1.8.2 at /usr/bin/ninja
  ninja: Entering directory `build_test'
  [36/40] Generating scrcpy-server with a custom command.
  (not invoking gradle, since we are root)
  [39/40] Running all tests.
  1/5 test_cbuf OK 0.02 s
  2/5 test_control_event_serialize OK 0.07 s
  3/5 test_device_event_deserialize OK 0.07 s
  4/5 test_queue OK 0.07 s
  5/5 test_strutil OK 0.07 s

Ok: 5
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0

Full log written to /opt/scrcpy/build_test/meson-logs/testlog.txt

Configure project :server
Checking the license for package Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/licenses
Warning: License for package Android SDK Build-Tools 28.0.3 not accepted.

FAILURE: Build failed with an exception.

• What went wrong:
  A problem occurred configuring project ':server'.

  Failed to install the following Android SDK packages as some licences have not been accepted.
  build-tools;28.0.3 Android SDK Build-Tools 28.0.3
  To build this project, accept the SDK license agreements and install the missing components using the Android Studio SDK Manager.
  Alternatively, to transfer the license agreements from one workstation to another, see http://d.android.com/r/studio-ui/export-licenses.html

  Using Android SDK: /usr/lib/android-sdk

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 4s

[unclehowell]

I ran the command sdkmanager --licenses and in Android-Studio I changed the SDK from Android 10 (Q) to Android 9 (Pie), which eliminated the above error but produced the following:

> Configure project :server
Checking the license for package Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/licenses
License for package Android SDK Build-Tools 28.0.3 accepted.
Preparing "Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)".
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" ready.
Installing Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/build-tools/28.0.3
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" complete.
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" finished.
Checking the license for package Android SDK Platform 29 in /usr/lib/android-sdk/licenses
License for package Android SDK Platform 29 accepted.
Preparing "Install Android SDK Platform 29 (revision: 3)".
"Install Android SDK Platform 29 (revision: 3)" ready.
Installing Android SDK Platform 29 in /usr/lib/android-sdk/platforms/android-29
"Install Android SDK Platform 29 (revision: 3)" complete.
"Install Android SDK Platform 29 (revision: 3)" finished.

> Task :server:compileDebugAidl FAILED
/usr/lib/android-sdk/build-tools/28.0.3/aidl: 1: /usr/lib/android-sdk/build-tools/28.0.3/aidl: Syntax error: redirection unexpected


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:compileDebugAidl'.
> com.android.ide.common.workers.WorkerExecutorException: 1 exception was raised by workers:
  java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: com.android.ide.common.process.ProcessException: Error while executing process /usr/lib/android-sdk/build-tools/28.0.3/aidl with arguments {-p/usr/lib/android-sdk/platforms/android-29/framework.aidl -o/opt/scrcpy/server/build/generated/aidl_source_output_dir/debug/compileDebugAidl/out -I/opt/scrcpy/server/src/main/aidl -I/opt/scrcpy/server/src/debug/aidl -d/tmp/aidl5784619688143858877.d /opt/scrcpy/server/src/main/aidl/android/view/IRotationWatcher.aidl}

Ended up going with VNC and Emteria until the snap version of ScrCpy (with the touchscreen fixes) is released. Keep up the great work

[sisco311]

@unclehowell, the snap from the edge channnel now is built from the dev branch.

[unclehowell]

You rock pal. Keep up the great work.