Libelektra: various python enhancements

[markus2330]

Some small nice to have for Python:

k==None comparison after lookup does not work.

```
ks = kdb.KeySet()
ks.append (kdb.Key("user/something"))
k = ks.lookup("user/something")
if k == None:
pass

expected outcome: k is not None, so "pass" should not be executed but the program should continue afterwards (it works if k != None)

actual outcome: exception ValueError: invalid null reference in method 'Key___eq__', argument 2 of type 'kdb::Key const &'

workaround: use `key is None` which is arguably the correct thing to do anyway



##  KeySet not printed

expected outcome: print key by key, similar to built-in data structures in Python
actual outcome:

>

workaround: write loop `for k in ks: print k,`



## Add key objects to hashmap

before c88d0b09414916f80c3eccdb376ad6a87805d867 it was not possible. Now it is possible but it might be buggy because users might change the name after adding the key to a hashmap. To avoid this, the readonly flag should be set. Maybe we even need a FrozenKey?

workaround: use the KeySet, which soon is a hashmap on lookups anyway

## No len(key)

import kdb
key = kdb.Key("user/hello/sub")
len(key)
Traceback (most recent call last):
File "", line 1, in
TypeError: object of type 'Key' has no len()

Expected behavior: return 3

workaround:

sum (1 for _ in key)
```

[markus2330]

@manuelm ping

[manuelm]

Some small nice to have for Python:

k==None comparison after lookup does not work.

expected outcome: k is not None, so "pass" should not be executed but the program should continue afterwards (it works if k != None)

actual outcome: exception ValueError: invalid null reference in method 'Key___eq__', argument 2 of type 'kdb::Key const &'

Fixed in SWIG 4: https://github.com/swig/swig/commit/687cf9c9c1b886ef4a828b2c29610183e9fef26d

KeySet not printed

No problemo.

Add key objects to hashmap

before c88d0b0 it was not possible. Now it is possible but it might be buggy because users might change the name after adding the key to a hashmap.

So what? Why is it disallowed that the hash changes? I see no problems with the current behavior.

No len(key)

Is there a C++ oder C function which returns the same value (I don't see any)? If not: why is this suddenly useful in the python context?

[markus2330]

Fixed in SWIG 4: swig/swig@687cf9c

:+1:

So what? Why is it disallowed that the hash changes? I see no problems with the current behavior.

If python handles this correctly and still finds the key, it is okay. Then we simply can add a test case for that.

Is there a C++ oder C function which returns the same value (I don't see any)? If not: why is this suddenly useful in the python context?

you can call std::distance on the NameIterator

[manuelm]

So what? Why is it disallowed that the hash changes? I see no problems with the current behavior.

If python handles this correctly and still finds the key, it is okay. Then we simply can add a test case for that.

k = kdb.Key("user/foo")
myset = set({ k, kdb.Key("user/bar") })
myset2 = set({ kdb.Key("user/foo"), kdb.Key("user/bar") })
print("myset=" + str(myset))
print("myset2=" + str(myset))
print("diff=" + str(myset.difference(myset2)))
k.name = "user/foo2"
print("myset=" + str(myset))
print("myset2=" + str(myset))
print("diff=" + str(myset.difference(myset2)))

myset={kdb.Key('user/bar'), kdb.Key('user/foo')}
myset2={kdb.Key('user/bar'), kdb.Key('user/foo')}
diff=set()
myset={kdb.Key('user/bar'), kdb.Key('user/foo2')}
myset2={kdb.Key('user/bar'), kdb.Key('user/foo2')}
diff={kdb.Key('user/foo2')}

Am I missing something?

Is there a C++ oder C function which returns the same value (I don't see any)? If not: why is this suddenly useful in the python context?

you can call std::distance on the NameIterator

I'll add it. Although I don't like this kind of operator overloading (abusing __len__).

[markus2330]

Am I missing something?

Someone might pick a name that already existed in the set.

I'll add it. Although I don't like this kind of operator overloading (abusing __len__).

Ok, thank you. You can also implement it another way. As background info: There is now this "new" API keyUnescapedName. It would be nice to have this feature (to be able to iterate over the parts of the names, and also to get the length, i.e. the number of null bytes in keyUnescapedName) also in python.

[manuelm]

Am I missing something?

Someone might pick a name that already existed in the set.

mh yeah. You're right:

k = kdb.Key("user/foo")
myset = set({ k })
k.name="user/foo2"
myset.add(kdb.Key("user/foo2"))
print("myset=" + str(myset))

myset={kdb.Key('user/foo2'), kdb.Key('user/foo2')}

Don't think we can do anything about this. Thinking more about this: Adding __hash__ to Key was wrong in the first place. It violates the constraints "never changes during lifetime": An object is hashable if it has a hash value which never changes during its lifetime (it needs a __hash__() method), and can be compared to other objects (it needs an __eq__() method). Hashable objects which compare equal must have the same hash value..

I'll add it. Although I don't like this kind of operator overloading (abusing len).

Ok, thank you. You can also implement it another way. As background info: There is now this "new" API keyUnescapedName. It would be nice to have this feature (to be able to iterate over the parts of the names, and also to get the length, i.e. the number of null bytes in keyUnescapedName) also in python.

I'll leave it as is.

[markus2330]

Exactly, if we provide __hash__, we need to freeze the key name (e.g. after __hash__ was called?), so that the hash cannot change anymore. This is exactly what was written in the top post. Alternatively, we could provide a FrozenKey, where it is impossible to change the name after construction (and only provide __hash__ there).

[markus2330]

Maybe we should also put keyLock to the public API. For FrozenKey, however, it would be enough to call keyNew with KEY_FLAG_LOCK_NAME

[manuelm]

I strictly advise against weird magic which "freezes" the key as soon as __hash__ gets called.

KEY_FLAG_LOCK_NAME sounds fine. __hash__ can check if the flag is set and return None otherwise. However checking the flag and throwing exceptions in all setters must be added in the cpp bindings.

However I do think this is beyond this issue and not really python related anymore.
... or you could just stop using sets with keys in the first place.

[markus2330]

I strictly advise against weird magic which "freezes" the key as soon as __hash__ gets called.

Yes, the statement also was with a big question mark.

KEY_FLAG_LOCK_NAME sounds fine. __hash__ can check if the flag is set and return None otherwise.

Sounds like a good idea.

However checking the flag and throwing exceptions in all setters must be added in the cpp bindings.

The checking of the flag already happens in C. All setters fail after the name was locked.

... or you could just stop using sets with keys in the first place.

It is good if Elektra plays with standard data structures nicely. They KeySet is not designed for any use case (e.g. it is not suitable if you want different orderings of keys).

[manuelm]

hmpf doesn't work that way. Python checks if the __hash__ of the class is set to None.

[markus2330]

Thank you very much!

Maybe @ulrikeschaefer even trapped into k==None comparison bug, unfortunately I did not see the stacktrace.