Libelektra: csvstorage: columns are ordered by name when using columns/index=name

[markus2330]

Steps to Reproduce the Problem

sudo kdb mount `pwd`/test.csv system/tests/csv csvstorage header=colname,columns/index=name
echo "b,a" > `kdb file system/tests/csv`
echo "valb,vala" >> `kdb file system/tests/csv`
kdb shell << HERE
kdbGet system/tests/csv
keySetName system/tests/csv/#2
ksAppendKey
keySetName system/tests/csv/#2/a
keySetString vala
ksAppendKey
keySetName system/tests/csv/#2/b
keySetString valb
ksAppendKey
kdbSet system/tests/csv
HERE

Expected Result

That I get the file:

b,a
valb,vala
valb,vala

Actual Result

a,b
vala,valb
vala,valb

System Information

• Elektra Version: master

Implementation hint

csvstorage needs to remember where the columns were via metadata "order".

[goJoWe16]

I would like to solve this bug as homework in the course CM, @markus2330 can you please assign me to this issue?

[markus2330]

Are you in contact with @vbalan92, who already chose an issue regarding the csvstorage (#2508)?

[goJoWe16]

No, not until now. Is this a problem? I can also do another issue if that is preferred.

[markus2330]

It might be a problem as refactoring in the plugin makes sense and this would not work without any coordination.

Let us see what @vbalan92 says but you can also search for other issues in the meantime.

[vbalan92]

Hi guys. As I am responsible for (#2508), I think I can also fix this issue.

[markus2330]

Ok, thank you! It makes a lot of sense that you fix all csvstorage related issues. Then you could make the whole homework around csvstorage.

[vbalan92]

The option 'columns/index=name' creates the following paths:

• 'system/tests/csv/(null)/a'
• 'system/tests/csv/(null)/b'
  because 'name' is not a column that exists in the file.
  Because of this '(null)' is used.

Is this how the plugin is supposed to work?

[markus2330]

If the plugins outputs (null) this is most likely a bug. (null) means that the value of a null-key was accessed, which should usually not be done.

Unfortunately, you did not specify all options, so it is unclear what you actually did but maybe you run into #2476 or #2518?

Please also document how the plugin works (not all interactions between options are described) and if it does not make much sense, please propose changes.

[vbalan92]

The option 'columns/index=name' creates the following paths:

• 'system/tests/csv/(null)/a'
• 'system/tests/csv/(null)/b'
  because 'name' is not a column that exists in the file.
  Because of this '(null)' is used.

Is this how the plugin is supposed to work?

The options I used are the once used for this issue:

• sudo kdb mount pwd/test.csv system/tests/csv csvstorage header=colname,columns/index=name

The (null) in the key is present because 'colums/index=name'. In this case we do not have a colum with this name, we only have the columns 'a' and 'b'.

When I try to fix the order of the colums (resolve the problem mentioned in this issue), I get the following keys:

• 'system/tests/csv/(null)/a'
• 'system/tests/csv/(null)/b'
• 'system/tests/csv/#2/a'
• 'system/tests/csv/#2/b'.

The first two are obtained when reading the already existing content of the file and the last 2 are present because of the following commands:

• keySetName system/tests/csv/#2/a
• keySetName system/tests/csv/#2/b.

To sum up, is the presence of the (null) in the key path considered a bug or not?

[markus2330]

Thank you for the elaborate explanation.

Yes, the presence of (null) is definitely a bug.

If the name is not present, it is maybe a user error (the user might have mistyped the name), so yielding a warning makes sense (or an error if the data cannot be written at all or would cause loss of data). I think we could simply use the array indexes, starting from #0, #1, ..., #_10... (so basically ignore the options that cannot be applied.) Does this make sense for you?

[stale[bot]]

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue.
Thank you for your contributions :sparkling_heart:

[stale[bot]]

I closed this issue now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue.
Thank you for your contributions :sparkling_heart: