Visidata: [2-5] cursorColIndex does not provide accurate index

[geekscrapy]

Description
When using cursorColIndex my instinct is that it should provide the column index. It seems to do this correctly unless the col is a keycol.

Scenario
You have many columns, some of which are offscreen. You scroll to the right (previously off screen) and make a couple of these (previously) off screen columns, keycols.

Example
If you use a command on one of these new keycols and the addColumn function sets the index arg like addColumn(Column(), index=cursorColIndex+1) (e.g. addcol-expr, or capture-col) - this new column gets created off screen, as (I think) the cursorColIndex is from the columns previous location, rather than its current (a key col). This makes the user think two questions:

1. did I create the col successfully?
2. where is my new column?

Expected result
If I create new cols based off of keycols (I usually use keycols to highlight things of interest or so I can see particular values quickly), I also want to have any new cols that are created off of a keycol to be placed next to the keycol edge (basically as close to the original column as possible).

I think this is low priority, but for new users this may create confusion (as it did for me recently) as they dont know if the column was created (if the original column wasn't in an on screen position), and if they even ran the command correctly.

Potential fixes

1. A quick fix for this might be to issue a vd.warning to highlight that a new col has been created offscreen
2. have cursorColIndex reflect the true order of columns, including keycols

[saulpw]

These should always be true:

• columns[cursorColIndex] == cursorCol
• visibleCols[cursorVisibleColIndex] == cursorCol

I think what you want, is for addColumn to take a visibleColIndex, instead of a colIndex. We should review all uses of addColumn that use index=, and see if this is a reasonable API adjustment--it might be better all around.

[geekscrapy]

Should I be able to see cursorVisibleColIndex from pyobj-sheet? I see other related values.
Or is there a way to get these values at runtime for debugging?

[saulpw]

If you use g Ctrl+Y to open the current sheet object as a pyobj-sheet, both of those should be there. You can also set a status option to include {sheet.cursorColIndex} etc.

[geekscrapy]

I was being dense - yes, they are. Sorry

[geekscrapy]

Ok, so a bit of testing later I think I understand what my expectations mean... Maybe.

Using cursorColIndex for any column which isn't a keycol is fine, you can use cursorColIndex+1 as the value for index in addColumn and it adds in the expected location, with and without any previously hidden columns. As noted in the OP, this location is often unexpected, so therefore cursorVisibleColIndex is advised when the source column is a keycol.

However, when using cursorVisibleColIndex on non-keycols and with sheets with hidden columns, the index value (cursorVisibleColIndex+1) doesn't make sense and the column is created cursorVisibleColIndex-[number of hidden columns] away from the original column.

Therefore I think the confusion comes with the addColumn index arg. I didn't really take much notice before, but this is probably referring the absolute column index rather than visibleCols index if I'm correct? Maybe there should be a couple of optional args for addColumn, visibleColIndex and absoluteColIndex?

Unless there is a quick way to get visibleColIndex+1 when there are hidden columns?

[geekscrapy]

FYI - this is working to provide a reliable index: cursorVisibleColIndex if cursorCol.keycol>0 else cursorColIndex

[saulpw]

If you have 2 keycols, and do an addcol on the first one, would you expect the added column to be a key column? or would it be added as the leftmost non-key column?

[geekscrapy]

I'd say, as a keycol. Reasoning - your keycol is something of interest, therefore if you create a new column from that keycol, it's likely to be something of interest too. 3 other examples/thoughts:

An example of this was I had 3 keycols of dates. I wanted to do a gFreq over them to see the most common dates (YYYYmmdd). The resolution and disp_date of these date cols were to the second. I didn't want to change the disp_date as I wanted the resolution, so I created 3 more columns using expr=col1.date(), these were all created on the far right each time and was quite a few clicks before I could get them back as keycols. On the flip side, if they had been created next to their original columns, a lot less clicks.

You still might loose new columns if you have many (or just wide) key cols if they are placed outside of the keycol space.

It's lot less clicks to remove a new column from keycol space, than to navigate to, re-add, and maybe reorder the a new column.

[saulpw]

okay, thanks for sharing your opinion and reasoning, @geekscrapy, that's helpful. I gave it a lot of thought, and I think the answer here is to add a new function, addColumnAtCursor. 99% of the uses of cursorColIndex were for this purpose, so I think we can probably deprecate cursorColIndex itself. I made it so that if the cursor is on a key column, the new column will not be a key column, but will show up directly after all the key columns. If you only have 1 key column, it should be seamless, and fix this issue. New columns won't be key columns, but I think that's more consistent with how key columns are treated already.