H5py: Unable to create file on CentOS (unable to lock file, errno = 37, error message = 'No locks available')

[jerermyyoung]

Hi everyone
I'm using Python 3.6.3 on CentOS 7.4.1108. My h5py version is 2.8.0 with HDF5 version being 1.10.2 and numpy 1.14.5. When I trying to create a file following the tutorial, I encountered with such problem

import h5py
import numpy as np
with h5py.File("mytestfile.hdf5", "w") as f:
... dset = f.create_dataset("mydataset", (100,), dtype='i')
...
Traceback (most recent call last):
File "", line 1, in
File "/home/user02/opt/python3/lib/python3.6/site-packages/h5py/_hl/files.py", line 312, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/home/user02/opt/python3/lib/python3.6/site-packages/h5py/_hl/files.py", line 148, in make_fid
fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 98, in h5py.h5f.create
OSError: Unable to create file (unable to lock file, errno = 37, error message = 'No locks available')

I'm testing a remote server which used NFS file format so I wonder it's the file format that causes this problem. But I have no idea on how to solve it. Is there anyone having similar problems?

[vasole]

Hello,

We faced a similar (the same?) problem with an NFS mounted file system. At the client side of the NFS (where we were running h5py), we were able to workaround the issue by setting the environment variable HDF5_USE_FILE_LOCKING to the five-character string 'FALSE'. The actual fix came from one of our system administrators who changed some of the NFS parameters at the server side. Unfortunately that knowledge was not shared with us. If you are successful with the workaround you are very likely facing the same issue.

[jerermyyoung]

@vasole Seems that it works well on our server...Now I can create .h5 file on it. Thanks a lot and I think that this issue can be closed if nobody post the 'actual fix' that you mentioned.

[nishaChandramoorthy]

Dea` all,

I am seeing this issue as well with HDF5 version 1.10.0 and h5py version 2.8.0.
Here is a minimal example reproducing the error on a ZFS file system:

 import h5py 
 import numpy as np
 data_matrix = np.random.uniform(-1, 1, size=(2, 2))

# Write data to HDF5
data_file = h5py.File('file.hdf5', 'w')
data_file.create_dataset('group_name', data=data_matrix)
data_file.close()

filename = 'file.hdf5'
f = h5py.File(filename, 'r')

# List all groups
print("Keys: %s" % f.keys())
a_group_key = list(f.keys())[0]

# Get the data
data = list(f[a_group_key])
print(data)

The above code tries to write and read an HDF5 file and throws the same error with this stacktrace:

File "somepath/python2.7/site-packages/h5py-2.8.0-py2.7-linux-x86_64.egg/h5py/_hl/files.py", line 148, in make_fid fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 98, in h5py.h5f.create IOError: Unable to create file (unable to flock file, errno = 37, error message = 'No locks available')

Here are some fixes that I have found recommended here and/or on Stackoverflow that I have tried, to no
avail:

1. Setting HDF5_USE_FILE_LOCKING='FALSE'
2. (On the ZFS server) Mounting the ZFS file system with the noac option
3. (On the ZFS server) Having the service rpc_statd running
4. (On the ZFS server) Setting the ZFS property nbmand to on

I'd really appreciate any suggestions you may have!