Deeplabcut: .h5 file is not created after labelling frames for training (pytables)

[antdlc]

Your Operating system and DeepLabCut version
Windows 10, with an Anaconda Env, & DeepLabCut2.x.

Describe the problem
After labelling the frames of my first training video, a .h5 file is not created in the labelled data folder, only a csv file is created. The following error message appears:

`>>> deeplabcut.label_frames(config_path)
Working on folder: 10052708
Traceback (most recent call last):
File "D:\Conda\envs\dlcdependencies\lib\site-packages\deeplabcut\generate_training_dataset\labeling_toolbox.py", line 356, in save
self.dataFrame.to_hdf(os.path.join(self.dir,"CollectedData_" + self.scorer + '.h5'),'df_with_missing',format='table', mode='w')
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\core\generic.py", line 1299, in to_hdf
return pytables.to_hdf(path_or_buf, key, self, *kwargs)
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 279, in to_hdf
complib=complib) as store:
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 466, in __init__
self.open(mode=mode, *kwargs)
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 605, in open
self._handle = tables.open_file(self._path, self._mode, *kwargs)
File "D:\Conda\envs\dlcdependencies\lib\site-packages\tables\file.py", line 320, in open_file
return File(filename, mode, title, root_uep, filters, *kwargs)
File "D:\Conda\envs\dlcdependencies\lib\site-packages\tables\file.py", line 784, in __init__
self._g_new(filename, mode, **params)
File "tables\hdf5extension.pyx", line 489, in tables.hdf5extension.File._g_new
tables.exceptions.HDF5ExtError: HDF5 error back trace

File "D:\Build\HDF5\hdf5-1.10.2\src\H5F.c", line 445, in H5Fcreate
unable to create file
File "D:\Build\HDF5\hdf5-1.10.2\src\H5Fint.c", line 1461, in H5F_open
unable to open file: time = Sun Jan 6 17:01:02 2019
, name = 'path\labeled-data\10052708\CollectedData_lina.h5', tent_flags = 13
File "D:\Build\HDF5\hdf5-1.10.2\src\H5FD.c", line 733, in H5FD_open
open failed
File "D:\Build\HDF5\hdf5-1.10.2\src\H5FDsec2.c", line 346, in H5FD_sec2_open
unable to open file: name = 'path\labeled-data\10052708\CollectedData_lina.h5', errno = 2, error message = 'No such file or directory', flags = 13, o_flags = 302

End of HDF5 error back trace

Unable to open/create file 'path\labeled-data\10052708\CollectedData_lina.h5'
`

How to Reproduce the problem
enter deeplabcut.label_frames(config_path)
label frames

Please advice / let me know if you need additional information. Thanks!

[MMathisLab]

Can you print what the full config_path variable is? Seems you didn’t add the full path based on the error message.

(Unable to open file: name = 'path\labeled-data\10052708\CollectedData_lina.h5',)

[antdlc]

Thanks for the quick reply! I am not using my own PC hence I changed the path name to post it on here. I have removed names now, it is:
D:\User\Name\DeepLabCut\ProjectName\ProjectName-lina-2019-01-06\labeled-data\10052708\CollectedData_lina.h5

[MMathisLab]

Then it’s likely a windows formatting error, please see the wiki at the top to look at troubleshooting tips; you need to format the path differently for Windows.

in Windows enter paths by either:

r`C:\Users\computername\Videos\reachingvideo1.avi'

`C:\Users\computername\Videos\reachingvideo1.avi' (i.e. double backslashes \ \ between C: and Users)

[antdlc]

When setting the config_path I used double backslashes. Extracting and cropping frames worked fine with this variable and running the demo beforehand wasn't a problem either.

[MMathisLab]

Okay, it wasn't clear from what you posted as what you set as config_path variable ...

Then you either don’t have admin privileges (ie pandas can’t create the file), or your it’s a pandas error. Ie this isn’t a known DLC error.

[antdlc]

I do indeed not have admin privileges, but it all worked without problems when I was running the demo on the same PC with the same missing privileges.

[MMathisLab]

Well, I’m not sure then. If the demo works, then DLC is running and installed correctly. I would just carefully check what differences there are in your set up vs the demo... (that is hard to troubleshoot via GitHub, sorry!)

[antdlc]

I will try again tomorrow then, using a different video, maybe it has to do with the video file. Thank you very much for your help, have a nice evening :)

[MMathisLab]

(Last thought, it could still be admin privileges, as the demos already give you labeled data, so I would right click on “cmd” and run as an admin)

It wouldn’t be the video as the h5 is made from the png images, so it’s something about the creating a h5...

[AlexEMG]

Yesterday, I saw a (slightly different) but also hdf file related error. Perhaps downgrade to pytables 3.4.3. You can do this in the environment by typing:

conda install pytables==3.4.3

[antdlc]

Tried that, but now getting the following error:

deeplabcut.label_frames(config_path)
Working on folder: 10052708
Traceback (most recent call last):
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 447, in __init__
import tables # noqa
File "D:\Conda\envs\dlcdependencies\lib\site-packages\tables__init__.py", line 131, in
from .file import File, open_file, copy_file
File "D:\Conda\envs\dlcdependencies\lib\site-packages\tables\file.py", line 35, in
from . import hdf5extension
ImportError: DLL load failed: The specified procedure could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Conda\envs\dlcdependencies\lib\site-packages\deeplabcut\generate_training_dataset\labeling_toolbox.py", line 356, in save
self.dataFrame.to_hdf(os.path.join(self.dir,"CollectedData_" + self.scorer + '.h5'),'df_with_missing',format='table', mode='w')
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\core\generic.py", line 1299, in to_hdf
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 279, in to_hdf
complib=complib) as store:
File "D:\Conda\envs\dlcdependencies\lib\site-packages\pandas\io\pytables.py", line 450, in __init__
'importing'.format(ex=str(ex)))
ImportError: HDFStore requires PyTables, "DLL load failed: The specified procedure could not be found." problem importing

(unfortunately I don't have the possibility to run as admin as I'm using a PC where I can't get admin rights)

[MMathisLab]

Ah, then unfortunately it won’t work; it requires admin rights to create files.... you can double check if you enter terminal/command you “open as admin”...

Also looking at the pytables GitHub issues, I would not use python 3.7; be sure your environment was created with python 3.6. You can use the supplied conda file from our GitHub, and try that...

[AlexEMG]

Unfortunately, this looks like a broken pytables/hdf installation.

[antdlc]

I tried uninstalling pytables but an error occurred, so after not being able to fix that, I uninstalled and then reinstalled deeplabcut. When trying to label images now, it does'nt come up with the pytables-error anymore but it's back to the original hdf error.

[MMathisLab]

last attempt may be to try the fix in #63 conda install -c conda-forge pytables (otherwise it's like admin issue....) I'm going to close for now, but re-open if needed

[dlhagger]

I had this happen to me on Windows as well (most recent DLC). conda install pytables worked to fix it, but I had to delete the project file and start over to get the labeled frames to save correctly.

[antdlc]

Turns out it was not an admin problem after all: I had a diacritic in my path, as soon as I removed that, all worked just fine.
Thanks for all your help!

[AlexEMG]

I had this happen to me on Windows as well (most recent DLC). conda install pytables worked to fix it, but I had to delete the project file and start over to get the labeled frames to save correctly.

@dlhagger : you can (usually) rescue the labels from the csv file by converting them (see https://forum.image.sc/t/how-can-i-solve-this-problem-importerror-hdfstore-requires-pytables/22555/3)