Termux-app: Terminal output speed issues

[evg-zhabotinsky]

This is probably the same as #218 and #368, but I felt that they might not describe the problem adequately.

Terminal output is too slow on many different occasions. When Termux cannot keep up with produced output, it stutters at best and becomes unresponsive for minutes at worst. As far as I can tell, using ANSI escapes (for colors, ncurses ui, etc.) makes things even worse.

Most typical manifestations in my experience:

• Scrolling in less and similar utilities. Scrolling down is fine, scronling up redraws whole screen and lags like hell.
• Build logs. Use minimal verbosity and you should be fine. Up verbosity a bit or get a lot of warnings due to a typo in a header, and your terminal is busted. Screen updates once every few seconds, and you'll be catching up with an already finished build for many minutes. (I usually just kill Termux and reconnect to my tmux session.)

This has been happening for a long while, half a year at very least. However, after one of the most recent updates (noticed on 0.60), the problem got much worse. On my Samsung tablet running Android 5.1.1, 80x25 terminal is mostly bearable, while on 190x40 vim takes _almost a minute_ to start and regularly hangs for a few seconds (e.g. on newline entry). Not nearly as bad on my phone running 6.0.1, but still noticeable.

I haven't yet looked at source code (don't have time right now, neither do I have Android app development experience), but I intend to unless it's fixed in a week or two. I suspect excessive syncing among pty and UI threads and maybe inefficient scrollback handling.

[xeffyr]

80x25 terminal is mostly bearable, while on 190x40 vim takes almost a minute to start and regularly hangs for a few seconds (e.g. on newline entry).

On Nexus 7 (Android 5.1.1) no such problem. It takes ~2 seconds to start vim on 182x53 and no lags when editing files. Are you installed some plugins ?
On Samsung Galaxy S7 it runs very fast without problems.

I never had lags when using Termux except when I connected over SSH with bad wifi signal.

[evg-zhabotinsky]

It takes ~2 seconds to start vim

That's already long enough. A heavily modded Vim (a vim-based IDE called Yavide) starts instantaneously (before I even release Enter key) on my laptop which has just slightly higher CPU frequency.

Also, the mere fact that terminal size noticeably affects startup time is a dead giveaway that something's wrong with terminal, 60x182x53x4 (60fps, each character is 2 bytes for unicode + 2 bytes for attributes) ~= 2MiB/s is a laughable bandwidth requirement unless you go over a slow network.

However, your point is valid in general. When I get home I will verify how fast Termux environment and Vim specifically work when accessed over SSH, thus bypassing terminal only and keeping everything else the same.

[Auxilus]

I haven't had the speed issue in termux, vim starts for me instantly... Though emacs takes some time loading..

[SDRausty]

There are lags at times. They are most likely from processes running in Termux invoked by the user. If these cause trouble, make them nice -n 20 as suggested.

[xeffyr]

I guess that this depends on a device. I created a big text file (68322 bytes) and tried to do a benchmark for 'cat' with command time cat textfile.txt. Results:
Nexus7:

real    0m1.063s
user    0m0.000s
sys     0m0.000s

Samsung Galaxy S7:

real    0m0.022s
user    0m0.010s
sys     0m0.000s

As you can see, even simple 'cat' command have a significant difference (1.063 vs 0.022, in seconds) in performance when executed in Termux terminal.

I will try to check performance of different Android terminal emulators.

[xeffyr]

Android Terminal Emulator (jackpal.androidterm) is faster. Results for the same text file are:
Nexus7:

real    0m0.181s
user    0m0.000s
sys     0m0.000s

And for Samsung Galaxy S7:

real    0m0.003s
user    0m0.000s
sys     0m0.000s

So, theoretically it should be possible to improve terminal throughput in Termux.

[SDRausty]

from processes running in Termux invoked by the user.

To any Android processes like apps, and even the base system that comes with Android. It sometimes uses a bit of CPU power. Usually just for a little while.

[evg-zhabotinsky]

Did the cat test just now on my phone:

cd /sdcard
dd if=/dev/urandom bs=1K count=1K | hexdump -C >big.txt  # ~5.5MiB plain text file
time cat big.txt >/dev/null  # 0.07s real time, thus disk read time is irrelevant

time cat big.txt  # Tested on different terminals:
# 1.30s -- tmux (inside Termux) when window is not in foreground
# 2.97s -- Android terminal by jackpal
# 38.9s -- CyanogenMod terminal. It is buggy, line ordering was messed up :/
# 67.6s -- Termux, same for active session, background session and minimized app

There definitely is a lot of room for improvement.
Also, note that this is on my phone where vim starts in a few of seconds on Termux. It takes a minute to do that on my tablet which isn't with me right now, so I expect much longer times when I run the test on it.

[SDRausty]

dd if=/dev/urandom bs=1K count=100 | hexdump -C >500KBfile.txt
for i in 0 1 2 3; do
        echo $i
        sleep 1
        time nice -n 20 cat 500KBfile.txt
        sleep 2
done

Termux

real    0m8.320s
user    0m0.010s
sys     0m0.100s

Arch Linux in Termux PRoot

real    0m8.669s
user    0m0.000s
sys     0m0.100s

aarch64 Android 6.1

[evg-zhabotinsky]

@sdrausty Could you please explain what you tried to measure? Also, whatever you tried to achieve with nice, it made no difference when I cated my file once more.
If you simply tried to compare different environments, cat is too primitive for that to matter. It just does read and write syscalls in a loop so all the real work happens inside kernel. Running cat big.txt >/dev/null finished instantaneously so writes to Termux pty are the bottleneck.

[SDRausty]

@evg-zhabotinsky thanks for the information and topic.

dd if=/dev/urandom bs=1K count=100 | hexdump -C >500KBfile.txt ; for i in {1..4}; do echo $i ; sleep 1 ; time nice -n 20 cat 500KBfile.txt ; sleep 2 ; done

Could you please explain what you tried to measure?

Standard output to terminal timing in Termux and Arch Linux in Termux PRoot. It's interesting to observe the lags. Why isn't the entire ouput fluid? The output stalls at times. Arch Linux in Termux PRoot uses an emulated kernel via Termux PRoot so the benchmarks should be different. They do not appear to be significantly different. The first result is in vanilla Termux. The second in Arch Linux in Termux PRoot.

whatever you tried to achieve with nice

Termux responsiveness while running the code.

       Run COMMAND with an adjusted niceness, which affects process
       scheduling.  With no COMMAND, print the current niceness.  Niceness
       values range from -20 (most favorable to the process) to 19 (least
       favorable to the process).

From man nice in Termux.

Running time cat big.txt crashed Termux (all sessions). nice -n 20 should keep crashing from happening to a large degree by moving the niced process to the background.

[evg-zhabotinsky]

@sdrausty proot simply uses ptrace syscall to intercept some other syscalls and fake chroot (and some other features too) by slightly altering syscall behavior (e.g. modifying file path). Host kernel is still used for everything.
When it comes to testing, the simpler the better, so simple cat with different terminals is the ideal test subject. Not sure why I didn't think of it myself.

[SDRausty]

@evg-zhabotinsky thanks for sharing about proot's behavior.

simple cat with different terminals is the ideal test subject.

time nice -n 20 cat 500KBfile.txt : nice -n 20 is customisable and optional. It is there for a reason, Termux and device behaviour while benchmarking. The variance in benchmark timings suggests multiple tests should be run. Vanilla Termux 33.4~35.8 on armv7l Android.

Host kernel is still used for everything.

Where does "emulated kernel" fit into the PRoot (chroot) picture?

simple cat with different terminals

Is there a way to open new sessions from a Termux session using the CLI? This question may be important in resolving https://github.com/termux/proot/issues/10 pacman -S archlinux-keyring Hangs in Arch Linux Termux PRoot #10 specifically

run some commands in a loop on another terminal

From https://github.com/termux/termux-packages/issues/2180#issuecomment-367881283

[SDRausty]

@xeffyr this benchmark is without --kernel-release=4.14.15 for exec proot …

Arch Linux in Termux PRoot Terminal Output Benchmark Result

real    0m7.846s
user    0m0.000s
sys     0m0.090s

Faster than vanilla Termux? Yes, faster by ~0.5s! See 0m8.320s above. Termux PRoot is about 6% faster than Termux. This is statistically significant. How is this possible?

[evg-zhabotinsky]

@sdrausty

1. There is no such thing as "emulated kernel" with chroot, proot, qumu-usermode, bsd jails and similar, including any combinations or derivatives of thereof.
2. We are getting off topic. The suspected issue is with terminal, so in order to prove or disprove that claim tests should use, ideally, exactly the same environment but different terminal emulators. You are trying to do the opposite.

As for the cat tests on my tablet (copied big.txt from my phone):

• 1.33s with tmux
• 68s with Jackpal terminal
• _1 hour_ with Termux (give or take a couple minutes, I forgot to disable screen timeout at first)

I guess there has to be something wrong with that tablet (~60x slowdown for Termux and ~20x for Jackpal compared to my phone), but the problem is on Android side and not with Linux base or Termux environment since tmux under Termux is fast on both devices. In any case, Termux is still much slower than it could be so my tablet should be a good test subject for worst case scenario.