Visidata: null data in input describe sheet

[aborruso]

Hi,
I have VisiData v2.-5dev.

Starting from this input file

a,b
,3
1,2
,4
5,5

when I open it, and then run shift + i I have

|column |errors |nulls |distinct |mode |min |max |sum |median |mean |stdev |
|--------|-------:|------:|---------:|------|-----|-----|-----|--------|-----:|------:|
|a |0 |0 |3 | | | | | | | |
|b |0 |0 |4 |#ERR | | | | | | |

Why do I have no nulls? If I add --null-value="" option I have the same output.

Thank you

[aborruso]

Am I the only one that has this behavior?

[anjakefala]

No @aborruso! This is a bug specifically with that option, but I want to fix it for all the options that have this problem, so it is going to take a bit longer to have a fix. =)

Current workaround is to prefix the option with -g to set it globally. The issue is that the function that is calculating whether a cell is Null is pulling from the global options, not from the sheet specific ones.

vd file.tsv -g --null-value=""

The original issue you ran into, where by default VisiData was not parsing those cells as None is expected!

[anjakefala]

This should now be fixed on develop @aborruso!

Thank you so much for this bug report. =)

[aborruso]

Hi @anjakefala ,
I have just launched python3 -m pip install --upgrade --user git+https://github.com/saulpw/visidata.git@develop.

If I open this file, running vd input.csv.

a,b
,3
4,
,3

In the describe sheet I have no null (also using --null-value=""). Am I doing something wrong?

Thank you

[anjakefala]

vd --null-value="" test.csv and vd test.csv --null-value="" work for me for that test case!

For it to look the same without the option set is expected.

edit: Oh, I am sorry, VisiData is recognising and marking them as Nones, but yeah, they are not showing up in the DescribeSheet as nulls. Okay, re-opening this issue until I understand what is happening.

[anjakefala]

LET'S TRY NOW =D (with the option!)

[aborruso]

It works, thank you!!

To have it as default, do I must add options.null-value="" to .visidatarc?

[anjakefala]

Yes. =)

[aborruso]

Yes. =)

I have tried it, but it does not work.

However in my opinion an empty cell should be makked as NULL one by default (as it was in vd 1.5).

Thank you

[anjakefala]

Sorry, it is options.null_value = ""! I was tired when I read this, and did not look at it closely. >_<

[saulpw]

However in my opinion an empty cell should be makked as NULL one by default (as it was in vd 1.5).

@aborruso Thanks for this suggestion. I need to give it some thought but you may be right.

[aborruso]

Sorry, it is options.null_value = ""! I was tired when I read this, and did not look at it closely. >_<

Thank you, it works