H5py: [BUG] track_order = True causes `RuntimeError: Can't rename attribute (record is already in B-tree)`

[1313e]

To assist reproducing bugs, please include the following:

• Operating System: Windows/Linux Ubuntu/MacOS-X
• Python version: 3.5+
• Where Python was acquired: Anaconda
• h5py version: 2.9.0+
• HDF5 version: 1.10.4
• The full traceback/stack trace shown (if it appears):

Traceback (most recent call last):
  File "h5py_bug.py", line 9, in <module>
    file.attrs[str(i)] = i
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/evandervelden/anaconda3/envs/py3/lib/python3.7/site-packages/h5py/_hl/attrs.py", line 100, in __setitem__
    self.create(name, data=value)
  File "/home/evandervelden/anaconda3/envs/py3/lib/python3.7/site-packages/h5py/_hl/attrs.py", line 217, in create
    h5a.rename(self._id, self._e(tempname), self._e(name))
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 118, in h5py.h5a.rename
RuntimeError: Can't rename attribute (record is already in B-tree)

---

This bug has been reported quite a few times already (e.g., #1180, #1185 and others probably), but all of them are reporting that the bug solely appears when building h5py against HDF5 1.10.5.
However, I can cause that bug to appear perfectly fine in HDF5 1.10.4 (and probably other versions as well), as long as the track_order parameter can be set to True (which was added in h5py 2.9.0).

The following code will reproduce this error on Python 3.5/3.6/3.7; on Windows/Linux Ubuntu/MacOS-X:

import h5py
h5py.get_config().track_order = True

with h5py.File('test.hdf5', 'a') as file:
    for i in range(20):
        print(i)
        file.attrs[str(i)] = i

After setting an attribute 8 times, setting the 9th will raise the error as shown above.
It does not matter how the file was opened; what the attributes are; what their types are; what is already contained in the HDF5-file; and so on.
It also does not matter how many attributes are set at once; setting the 9th will always trigger the error.
For example, the following will do the exact same thing:

import h5py
h5py.get_config().track_order = True

for i in range(20):
    with h5py.File('test.hdf5', 'a') as file:
        print(i)
        file.attrs[str(i)] = i

Attempting to execute either of these code snippets twice in a row (of course making sure that mode='a'), will cause the error immediately upon setting the first attribute.

Exception to the above is that when the code snippet is executed on an already existing HDF5-file that was created with track_order = False, it does not raise the error.
However, obviously, it will not keep track of the insertion order of any new attributes either.

Hopefully this helps.

[takluyver]

Just to double check, you have verified this with h5py 2.10, right?

[1313e]

Just to double check, you have verified this with h5py 2.10, right?

Yes

[wlievens]

I'm getting this on 2.10.0 whenever I enable track_storage and attempt to store 9 or more float64 attributes. Is there any progress on this issue?

[aragilar]

Interestingly, on latest h5py master, HDF5 1.10.6, py38 (Debian Unstable), I get a slightly different error when running the script twice (being unable to delete the attribute this time):

$ python test.py
0
1
2
3
4
5
6
7
8
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    file.attrs[str(i)] = i
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/aragilar/.virtualenvs/tmp-8398a5a08bbe44e/lib/python3.8/site-packages/h5py/_hl/attrs.py", line 96, in __setitem__
    self.create(name, data=value)
  File "/home/aragilar/.virtualenvs/tmp-8398a5a08bbe44e/lib/python3.8/site-packages/h5py/_hl/attrs.py", line 205, in create
    h5a.delete(self._id, self._e(name))
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 145, in h5py.h5a.delete
KeyError: 'Unable to delete attribute (record is not in B-tree)'

I wonder if this is a tracking issue, rather than a renaming issue (the next step would likely be to try to do this in C)?