Bat: Bat seems to alter icdiff output

[Lazza]

What version of bat are you using?

bat 0.15.4

Describe the bug you encountered:

I am not sure whether this is a problem with the icdiff tool or bat itself, however every time I pipe the former into the latter I get double line spacing.

Describe what you expected to happen?

The bat output should not have additional line spaces.

How did you install bat?

I used the DEB package provided in the GitHub releases.

---

system

$ uname -srm
Linux 5.4.0-47-generic x86_64

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

bat

$ bat --version
bat 0.15.4

$ env

bat_config

bat_wrapper

No wrapper script for 'bat'.

bat_wrapper_function

No wrapper function for 'bat'.

$ /usr/bin/fish --login -i -c functions cat

# Defined in - @ line 1
function cat --wraps=bat --description 'alias cat=bat'
bat  $argv;
end

tool

$ less --version
less 551 (GNU regular expressions)

[sharkdp]

Indeed, I can confirm this.

This is due to the way icdiff structures its output. It prints full-width lines and fills them with spaces:

> icdiff … | bat -Ap
␛[0;34ma␛[m••••••••••••••••••••••••••␛[0;34mb␛[m•••••••••••••••••••••••••␊
…

In addition, there are ANSI escape sequences for colorization. Most programs would disable this, if the output is piped. But I guess it makes sense for such a tool to keep them (if users want to pipe to a pager).

Somehow, bats wrapping mechanism handles this wrong and outputs an additional empty line:

───────┬──────────────────────────────────────────────
       │ STDIN
───────┼──────────────────────────────────────────────
   1   │ a                          b                 
       │         
   2   │ 6                          6                 
       │         
…

Looks like it doesn't skip the ANSI sequences?

@eth-p FYI.

[eth-p]

Thanks for bringing this to my attention. I'm going to be a bit busy with university this semester, but I'll try to come up with a fix during my spare time.

[sharkdp]

Just wanted to let you know if you are interested. And I certainly didn't want to call you in to fix it :-). Even if you would like to, there is absolutely no need for hurry. Never will be.

[Lazza]

Thank you very much for the feedback and debugging you performed.

IMHO it makes a lot of sense to keep the colors.

Is there a "quick" fix like stripping the trailing whitespace, without changing bat? It would probably be quite good for now, as a workaround.

Of course there is no need for hurry in any case. 🙂

[sharkdp]

A workaround for you could be to disable wrapping by adding --wrap=never to bats config file.

[Lazza]

@sharkdp I will get back to the Ubuntu machine in a few days and try then, thank you.

For now, I tried to use --wrap=never on macOS but it doesn't seem to produce any effect at all. Text seems to always wrap even if I call bat directly on a single file with long lines.

In the latter case I am using iTerm2 with the fish shell.

[eth-p]

I did some investigating, and it seems like icdiff is actually the one misbehaving. As a comparison, I ran icdiff ... | less -RN and ended up with an extremely similar result:

It appears that icdiff is using the full terminal width and filling it in with blank space characters, which then end up getting hard wrapped as part of bat's wrapping algorithm. I'm not sure that this is something we should try to fix, since that would end up with bat stripping trailing space characters which may have a semantic meaning (e.g. Rust multiline strings, or the entire Whitespace language).

[sharkdp]

Thank you for looking into this @eth-p! I agree. In this case, I don't think we should change anything in bat.

[Lazza]

Thank you for spending the time checking this and providing an explanation.

Can we maybe turn this into a feature proposal (optionally strip trailing white space by using a command line flag)?

Should I open a separate issue for the wrap=never that does not seem to work?

[eth-p]

Can we maybe turn this into a feature proposal (optionally strip trailing white space by using a command line flag)?

That would be fine, yeah. Any suggestions for the flag name?

Should I open a separate issue for the wrap=never that does not seem to work?

Yes, please.

[sharkdp]

Shouldn't this rather be left to an external tool then?

icdiff … | sed -r 's/[[:blank:]]+$//' | bat

I wouldn't want to introduce a new command-line flag just to fix a flaw in another program.

[Lazza]

OK thanks, I opened #1255 for the wrapping issue.

Regarding icdiff, I am currently running it like the following:

icdiff a.txt b.txt --cols=150 | bat

(my terminal width is 160)

Sorry for the waste of time.

---

Edit: here's a useful alias for fish...

alias diff='icdiff --cols=(echo (tput cols)"-8" | bc)'

[sharkdp]

Sorry for the waste of time.

No worries - thanks for reporting!