Bat: 'bat' gives different output than 'cat' when printing from IPython

[kbd]

A "bat has different behavior than cat" bug report (I have bat aliased to cat now):

⚡~/proj/stderr$ ipython -c "print('hello')"
hello
⚡~/proj/stderr$ ipython -c "print('hello')" | cat
───────┬───────────────────────────────────────────────────────────
       │ STDIN
───────┼───────────────────────────────────────────────────────────
   1   │ 0;IPython: proj/stderr^Ghello
───────┴───────────────────────────────────────────────────────────
⚡~/proj/stderr$ ipython -c "print('hello')" | \cat
hello
⚡~/proj/stderr$ python3 -c "print('hello')" | cat
───────┬───────────────────────────────────────────────────────────
       │ STDIN
───────┼───────────────────────────────────────────────────────────
   1   │ hello
───────┴───────────────────────────────────────────────────────────

I dunno what the 0;...^G syntax means or why IPython adds it, but figured I'd report.

Thanks again for bat!

[sharkdp]

Thank you for reporting this.

Interestingly, we can use bats new -A/--show-all option (#395) to figure out what's going on here. This option shows non-printable characters in a human-readable form:

/tmp                                                                             
▶ ipython -c "print('hello')" | bat -A
───────┬──────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼──────────────────────────────────────────────────────────────────────────
   1   │ ␛]0;IPython:•/tmp␇hello␊
───────┴──────────────────────────────────────────────────────────────────────────

Instead of just printing hello␊ (hello + line feed / new line), IPython first outputs ␛]0; (some form of escape sequence), then IPython: /tmp (I'm in the /tmp folder), then ␇ (BEL ASCII character). It turns out that these three pieces make up a Terminal control sequence to set the title of the terminal Window (see e.g. http://tldp.org/HOWTO/Xterm-Title-3.html). Indeed, if you just execute ipython -c "print('hello')", you should be able to see your terminal title change (just as long as the process is running).

IPython has a --no-term-title option to disable this behavior, so this works:

▶ ipython --no-term-title -c "print('hello')" | bat       
───────┬───────────────────────────────────────────────────────────────
       │ STDIN
───────┼───────────────────────────────────────────────────────────────
   1   │ hello
───────┴───────────────────────────────────────────────────────────────

(I'm slightly surprised that IPython does not use this setting when writing to a pipe)

Finally, there is still the question why this doesn't work when piping into bat. It turns out that this is actually a problem with the pager (less), and not bat itself. If we turn off paging, everything works fine:

▶ ipython -c "print('hello')" | bat --paging=never
───────┬───────────────────────────────────────────────────────────────
       │ STDIN
───────┼───────────────────────────────────────────────────────────────
   1   │ hello
───────┴───────────────────────────────────────────────────────────────

On the other hand, if you pipe the output of ipython into less, you will see the same garbled output.

It turns out that less has an option to fix this. If we use -r/--raw-control-chars instead of -R/--RAW-CONTROL-CHARS, everything works as expected:

▶ ipython -c "print('hello')" | bat --pager="less -FrX"
───────┬───────────────────────────────────────────────────────────────
       │ STDIN
───────┼───────────────────────────────────────────────────────────────
   1   │ hello
───────┴───────────────────────────────────────────────────────────────

Unfortunately, using -r has some downsides. From lesss man page:

Warning: when the -r option is used, less cannot keep track of the actual appearance of the screen (since this depends on how the screen responds to each type of control character). Thus, various display problems may result, such as long lines being split in the wrong place.

So in the end, I'm afraid we have to close this as "won't fix" :confused: (unless we can somehow show that using -r is fine)

See also:

• https://stackoverflow.com/questions/40046913/stop-ipython-from-changing-terminal-title

[sharkdp]

@kbd: Any update on this? It looks like this has been addressed upstream.

[kbd]

Hi @sharkdp. I have not gotten to look into the ipython bug like I wanted to (and said I would in that thread) but this is definitely not a problem in bat so this should be closed.

However, the question remained whether you can use -r (vs -R) in less and have it be ok, but I don't know anything about that. Are there test cases that break in one or the other? FWIW, -r does make this case work properly:

⚡~$ export BAT_PAGER="less -RFX"
⚡~$ ipython -c "print('testing')" | bat
───────┬──────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼──────────────────────────────────────────────────────────────────────────────────
   1   │ 0;IPython: Users/kbd^Gtesting
───────┴──────────────────────────────────────────────────────────────────────────────────
⚡~$ export BAT_PAGER="less -rFX"
⚡~$ ipython -c "print('testing')" | bat
───────┬──────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼──────────────────────────────────────────────────────────────────────────────────
   1   │ testing
───────┴──────────────────────────────────────────────────────────────────────────────────

[sharkdp]

However, the question remained whether you can use -r (vs -R) in less and have it be ok, but I don't know anything about that.

I don't know much about it either. One major problem is that we have to deal with multiple versions of less, some of which behave completely different. It would be great if there was a sane cross-platform pager (library) that we could ship with (use in) bat.

I am going to close this for now, as I don't see any way to proceed here.