Here's what I'm currently doing:
import h5py as h5
import pandas as pd
def init_stores():
"""Check if h5 exists, create one if it doesn't"""
for store_path in store_paths:
try:
with h5.File(store_path, 'r'):
pass
except OSError:
with h5.File(store_path, 'w'):
logger.info("Created h5 file: {}".format(store_path))
pass
def update_hdf():
with pd.HDFStore(store_path) as store:
print("In the update function")
exit()
And I'm getting this error:
Traceback (most recent call last):
File "C:\venv\lib\site-packages\pandas\io\pytables.py", line 445, in __init__
import tables # noqa
File "C:\venv\lib\site-packages\tables\__init__.py", line 90, in <module>
from .utilsextension import (
ImportError: DLL load failed: The specified procedure could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/.../hdf_stuff.py", line 93, in <module>
main()
File "C:/.../hdf_stuff.py", line 89, in main
update_hdf()
File "C:/.../hdf_stuff.py", line 39, in update_hdf
with pd.HDFStore(store_path) as store:
File "C:\venv\lib\site-packages\pandas\io\pytables.py", line 448, in __init__
'importing'.format(ex=str(ex)))
ImportError: HDFStore requires PyTables, "DLL load failed: The specified procedure could not be found." problem importing
If I comment out the h5py import, it works.
Can you fill in your #stuff
with what actually causes the failure? If I just be pass
there, I don't get an exception.
Can you also include your version info + your h5py and HDF5 versions?
@TomAugspurger all I have is a print statement (edited) and it fails.
Versions:
python 3.6
pandas 0.22.0
h5py 2.7.1
I'm not sure how to see the HDF5 version. I've only used what comes with pandas.
OK, your edited version does not fail for me, so it's presumable an environment thing.
What version of pytables?
On mac / linux h5cc -showconfig
will print out the HDF5 version. Pandas doesn't bundle HDF5. How'd you install pytables?
pip show tables
Version: 3.4.2
Installed with pip I believe
OK, I'm not sure if / how pytables or h5py wheels bundle HDF5.
On Mon, Feb 12, 2018 at 4:54 PM, Brendan Martin notifications@github.com
wrote:
pip show tables
Version: 3.4.2Installed with pip I believe
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/19666#issuecomment-365091390,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQHIn3FxZe5e9N9NxfYTmQJnn_hIWcHks5tUMErgaJpZM4SCwgl
.
please read the docs
HDFStore only works with pytables and has no support with h5py at all
you cannot use h5 files at all - maybe we need a bug warning in the docs - have seen a couple of issues like this in last few months
The issue isn't about using h5py with HDFStore, it's about some kind of
process-level conflict between pytables (plus or minus something pandas
does) and something h5py does on import.
On Mon, Feb 12, 2018 at 5:22 PM, Jeff Reback notifications@github.com
wrote:
you cannot use h5 files at all - maybe we need a bug warning in the docs -
have seen a couple of issues like this in last few months—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/19666#issuecomment-365097615,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQHIkJ0P02AGp-fyBnc8il4W13XjO3vks5tUMfIgaJpZM4SCwgl
.
this was an issue ages ago, but IIRC has been resolved in newer builds of hdf5. In any event this is not a pandas issue, rather an install / conda / pip one.
Yeah, I vaguely remember that too, and the fact that I can't reproduce it
points pretty strongly at an environment issue.
On Mon, Feb 12, 2018 at 6:16 PM, Jeff Reback notifications@github.com
wrote:
this was an issue ages ago, but IIRC has been resolved in newer builds of
hdf5. In any event this is not a pandas issue, rather an install / conda /
pip one.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/19666#issuecomment-365107952,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQHItjM3x049K4X_ICGyhOEFJ7q4ALbks5tUNRdgaJpZM4SCwgl
.
@TomAugspurger @jreback thanks for the help. Should I try to reinstall or just use pytables, or is there another solution? I'm not partial to using h5py since this is the first use in my project.
You might try conda. I know there was an issue with conda packages in the
past but I believe they were fixed.
Otherwise, you may need to restructure your program to avoid importing h5py
and pandas in the same process.
Let us know if you find anything out.
On Tue, Feb 13, 2018 at 9:01 AM, Brendan Martin notifications@github.com
wrote:
@TomAugspurger https://github.com/tomaugspurger @jreback
https://github.com/jreback thanks for the help. Should I try to
reinstall or just use pytables, or is there another solution? I'm not
partial to using h5py since this is the first use in my project.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/19666#issuecomment-365292645,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQHIsWglMlEBs8OJkPXyLPJ3kxGv6Utks5tUaPRgaJpZM4SCwgl
.
@TomAugspurger I'm just using pytables open_file
instead. Thanks for the help.
I managed to get pytables and h5py working together on windows10 by downgrading h5py to version 2.7.0.
pip uninstall h5py
pip install h5py=2.7.0
I suddenly started facing this issue. I did an uninstall followed by an install, and it fix the problem.
pip uninstall h5py
pip install h5py
The problem is with h5py packaged with Anaconda. I started with a fresh installation and still get the same error. Shame on Anaconda for not fixing bugs for months!
@nirvana1122 it's most likely not an issue with the conda package. If you have an example showing otherwise I'm sure the Anaconda team would appreciate a report.
Most helpful comment
I managed to get pytables and h5py working together on windows10 by downgrading h5py to version 2.7.0.
pip uninstall h5py
pip install h5py=2.7.0