Kakoune: Can't accurately select a tab character

[lenormf]

Steps

• printf '\t\t\t\t\t' | kak
• click on the last tab character

Outcome

Tab characters are selected one after the other (starting with the beginning of the line) until the one under the cursor actually is.

Expected

The tab character under the cursor should be selected on click.

[andreyorst]

This happens with any white space character:

(sorry for glitches)

Interesting though is that clicking before cursor jumps to first white-space in the area

[Screwtapello]

It's more dramatic when dragging a selection: left-to-right it feels mostly OK, but while dragging right-to-left the Kakoune cursor still moves left-to-right to meet the mouse cursor.

Exactly the same thing happens via the JSON UI, and mouse events seem to work perfectly on non-whitespace characters, so this seems to be a Kakoune problem rather than an ncurses problem.

[Screwtapello]

@shachaf mentioned this on IRC and I investigated further. Here's a cleaner test-case:

1. Launch Kakoune with the show-whitespaces highlighter, and a buffer containing whitespace:

   printf "%20sx" "" | kak -n -e "addhl global/ show-whitespaces"
   

2. Using the mouse, click about half-way between the left-hand edge of the terminal and the x.

Expected results:

• The selection moves to the space underneath the mouse cursor
• As the mouse is dragged, the selection follows it closely

Actual results:

• The selection moves a couple of characters toward the mouse cursor
• With repeated clicks or some dragging, the selection eventually catches up with the cursor
• While dragging, if the mouse-cursor moves left of the text-cursor, the text-cursor snaps back to the anchor. If the mouse-cursor moves left of the anchor, the text-cursor leaps to the beginning of the line

Notes:

• Everything works as expected without the show-whitespaces highlighter.

[Screwtapello]

Today I spent some time tracing through the code, and I think I've figured out what's going on.

When you click on some cell in the terminal, Kakoune has to figure out what span of the buffer produced that cell, so it can set the selection coordinates. Figuring out what line the cursor is on is pretty easy (at least until multi-line replacement highlighters were added, which are still glitchy: #3644), but the meat of the logic occurs in find_buffer_coord():

https://github.com/mawww/kakoune/blob/644660f65f42c25c594cbd804efabcb16bb67242/src/window.cc#L289-L309

It's called with line as a list of display atoms (that is, strings with formatting attributes), buffer as the underlying buffer being displayed, and column being the terminal column whose byte-offset we're trying to discover.

A DisplayAtom can be one of three types: Range directly represents a span of the underlying buffer, Text is wholly novel content (like the :info panel or the line-numbers highlighter), and ReplacedRange represents a span of the underlying buffer with a span of different text (like the replace-ranges highlighter). In the above code:

• Text atoms are completely ignored. That makes sense; they're not in the buffer so you can't click on them.
• Range atoms are "normal" (line 298-302): if column falls inside a Range atom, the resulting coordinate is column UTF-8 code-points along from the coordinate of the beginning of the range. Again, that makes perfect sense.
• ReplacedRange atoms are hand-waved (line 303): they represent some span of the buffer, but there's no guarantee that offset X into the replacement represents any part Y of the buffer, so we shrug and just say it's the character at the beginning of the atom.

This explains why the selection "walks" cell-by-cell toward the mouse-cursor when repeatedly clicking on the whitespace at the beginning of the line. Let's say a line begins with 8 spaces, the selection is on column 1, and the user clicks on column 4.

• The first space in the line is replaced by the show-whitespaces highlighter, and coloured with the primary selection colours, making it a display atom of its own. Spaces 2-8 are also replaced by the show-whitespaces highlighter, but they are the same colour so they share an atom.
• The user clicks on column 4.
• Upon entering find_buffer_coord(), we test the first atom. It's 1 column wide, and we want column 4, so we skip it.
• We test the second atom. It's 7 columns wide, and we want column 4, so we examine it.
• It's a ReplacedRange atom, so we snap the coordinate to the beginning of the range, column 2.
• Kakoune moves the selection to column 2, since it believes that's what the user clicked on.
• Now the first space in the line is coloured normally, so it's an atom of its own. The second space in the line is coloured with the primary selection, so it's also an atom of its own. Spaces 3-8 are coloured normally, so they're a third atom.
• The user clicks on column 4.
• When find_buffer_coord() checks the first atom, column 4 is not inside it, so we skip it.
• We test the second atom. Column 4 is still not inside it, so we skip it.
• We test the third atom. Column 4 is inside it, so we snap the coordinate to the beginning of the range, column 3.
• Kakoune moves the selection to column 3, since it believes that's what the user clicked on.
• etc.

I'm not sure what the best way forward is, here. Specifically for the case of showing whitespaces, it's pretty obvious - each character of the replacement corresponds to a single character of the buffer, so there's a reliable mapping back to buffer coordinates. However, the ReplacedRange system is more general: what if you tell Kakoune to display every instance of "cloud" in the buffer as "my butt" and the user clicks on the first "t"? Maybe Kakoune could interpolate coordinates, so a click 50% of the way along an atom is mapped to the character 50% of the way along the underlying buffer span, but a string like \t🐄 is mostly-tab in the terminal and mostly-cow in UTF-8, so there's no good way to map an arbitrary fraction between the two.