Pandas: ImportError: DLL load failed with Windows wheel for 1.0.2 and 1.0.3

[J-Postma]

When trying to install pandas 1.0.2 or newer versions on a docker, I run into the following import error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 55, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 60, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 124, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4572, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10349, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed while importing aggregations: The specified module could not be found.

I've reproduced this error on a separate pc by running:

docker run -it python cmd.exe
pip install pandas
python
import pandas

This error only occurs when trying to import pandas from the docker. Python and pip versions:

python 3.8.2
pip 20.0.2

[TomAugspurger]

Things work for you with pandas 1.0.1 and earlier?

[J-Postma]

Yes, things were fine with pandas 1.0.1 or any of the 0.* versions

[TomAugspurger]

Thanks. We switched our wheel building infrastructure for 1.0.3, but if it's happening on 1.0.2 as well that's likely not it.

I'm not aware of anyone else using pandas inside a windows docker container. You might be on your own here, so anything you can do to debug this would be welcome.

[TomAugspurger]

Can you create a new environment and post the full output of this?

pip install pandas
python -c "import pandas"

For both pandas 1.0.2 and 1.0.3 potentially.

[AaryamanCodes]

So, I too faced the exact error. I tried it on Jupyter notebooks and it worked though I could still not import some other pandas submodules.

[TomAugspurger]

From @mborus:

_TLDR; 2 DLLs are missing that were included in 1.0.1_

Code Sample, a copy-pastable example if possible

import pandas as pd

Problem description

I'm using Windows 7, German 64bit with a fresh installation of Python 3.8.2 64bit.
I updated pip, created a fresh virtual environment with "py -3 -m venv venv" and activated it.
This installation is in a restricted area with no direct, unfiltered internet access.

I installed Pandas 1.0.3 from a via "python -m pip download pandas" download made on an unrestricted similar Windows7 machine. I moved the download folder behind a firewall and installed Pandas like this: "python -m pip install pandas --no-index --find-links /path/to/downloads":

Pandas installs fine. It crashes on import with the error below (The last sentence says in English: The requested Module was not found)

As recommended by this stack overflow thread (https://stackoverflow.com/questions/60763529/unable-to-import-pandas-pandas-libs-window-aggregations) going back to 1.0.1 fixes the problem. (More below, after the error)

Error:

(venv) C:\my_program>python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import pandas as pd
Traceback (most recent call last):
File "", line 1, in
File "C:\my_program\venv\lib\site-packages\pandas__init__.py", line 5
5, in
from pandas.core.api import (
File "C:\my_program\venv\lib\site-packages\pandas\core\api.py", line 2
9, in
from pandas.core.groupby import Grouper, NamedAgg
File "C:\my_program\venv\lib\site-packages\pandas\core\groupby__init_
_.py", line 1, in
from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGr
oupBy
File "C:\my_program\venv\lib\site-packages\pandas\core\groupby\generic
.py", line 60, in
from pandas.core.frame import DataFrame
File "C:\my_program\venv\lib\site-packages\pandas\core\frame.py", line
124, in
from pandas.core.series import Series
File "C:\my_program\venv\lib\site-packages\pandas\core\series.py", lin
e 4572, in
Series._add_series_or_dataframe_operations()
File "C:\my_program\venv\lib\site-packages\pandas\core\generic.py", li
ne 10349, in _add_series_or_dataframe_operations
from pandas.core.window import EWM, Expanding, Rolling, Window
File "C:\my_program\venv\lib\site-packages\pandas\core\window__init__
.py", line 1, in
from pandas.core.window.ewm import EWM # noqa:F401
File "C:\my_program\venv\lib\site-packages\pandas\core\window\ewm.py",
line 5, in
import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed while importing aggregations: Das angegebene Modul
wurde nicht gefunden.

I compared the installation of version 1.0.1 to 1.0.3;

The 1.0.3 installs these files in
"\venv\Lib\site-packages\pandas_libs\window\"

    __init__.py
    aggregations.cp38-win_amd64.pyd
    indexers.cp38-win_amd64.pyd

The 1.0.1 version also installs these two DLLs in the same folder.

    concrt140.dll
    msvcp140.dll

When I manually place these DLLs into the above folder of the 1.0.3 version,
the import works without an error.

Expected Output

No error message

Output of pd.show_versions()

Not available, since pd did not import

[TomAugspurger]

To everyone in this issue, does the problem occur using wheels from pandas 1.0.2 and 1.0.3?

[mborus]

Can't say for 1.0.2, but I have installed the 1.0.3 wheel "pandas-1.0.3-cp38-cp38-win_amd64.whl" and extracted the DLLs from "pandas-1.0.1-cp38-cp38-win_amd64.whl"

[TomAugspurger]

@mborus can you try for 1.0.2? Our wheel building infrastructure changed for 1.0.3, but we've had users report issues with 1.0.2 as well.

[mborus]

I'll give it a try. This may take a while because the process of getting the Pandas wheel behind the firewall is a bit complicated. I'll post the result when I know more.

[TomAugspurger]

Thanks!

[mborus]

I installed the 1.0.2 (pandas-1.0.2-cp38-cp38-win_amd64.whl) briefly and it's the same problem. The two DLLs are not installed, so the import fails the same way.

[TomAugspurger]

OK thanks for checking!

@jbrockmendel or @WillAyd right now I'm guessing this is a wheel building issue, but do you know if we've had any changes to our build configuration between 1.0.1 and 1.0.2 that would cause this? https://github.com/pandas-dev/pandas/issues/32857#issuecomment-602787448 is the clearest summary.

[TomAugspurger]

Hmm https://github.com/pandas-dev/pandas/pull/32386/files touched window/aggregations. Is there anything there that seems off? It seems innocuous.

[AaryamanCodes]

I installed it using pip. I am using windows 10. Should I go to Panda's official website and download it?

[AaryamanCodes]

@TomAugspurger I am unable to use the pandas module though I can use it on Jupyter Notebooks without an error. How do I need to download any other version because I have used pip install.

[TomAugspurger]

@AaryamanCodes this isn't something that would be affected by using it through the notebook or through the terminal. Most likely your notebook is using a different environment.

It sounds like pandas 1.0.1 is not affected, so you can pin to that version for now.

[TomAugspurger]

cc @cgohlke if you have any guesses. (see https://github.com/pandas-dev/pandas/issues/32857#issuecomment-602787448). Though the fact that it's failing with your wheels and the one we built on azure pipelines indicates that it's a problem with our configuration.

[cgohlke]

I think the C++ runtime DLLs need to be packaged in the _libs/window directory :

https://github.com/pandas-dev/pandas/blob/f20331d543c5d42e54dff29135146fc0e6798a84/setup.py#L760

[TomAugspurger]

Thanks. Is it strange that this is only coming up for 1.0.2 and newer, since there weren't any changes to the setup.py on the 1.0.x branch after 1.0: https://github.com/pandas-dev/pandas/commits/1.0.x/setup.py

Can people running into this confirm that it only occurs with Python 3.8 (not Python 3.7)? 3.8 changed how DLL loading works IIRC. I've seen two people report using 3.8.2. Does anyone see it with 3.8.1?

[jaydave1988]

I am facing the same error for 1.0.3 and 1.0.2 using python 3.6.2.
C:\Users\Administrator>python --version
Python 3.6.2

[TomAugspurger]

OK thanks. So it probably isn't related to Python 3.8's changes to DLL loading then.

Is there a windows equivalent of ldd or otool? If I run

$ otool -L pandas/_libs/window/aggregations.cpython-38-darwin.so
pandas/_libs/window/aggregations.cpython-38-darwin.so:
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Can people check the windows equivalent for aggregations.pyd for 1.0.1 and 1.0.2 (or 1.0.3)?

Any if anyone is able to compile pandas from source, @cgohlke's suggestion https://github.com/pandas-dev/pandas/issues/32857#issuecomment-603279041 might provide a fix, even if we don't understand the cause.

[mborus]

One more thing about this: I looked close at the other Windows7 machine, where version 1.0.3 worked fine. On this machine I found the missing DLLs msvcp140.dll and concrt140.dll on the system path in c:\windows\system32 and dozens of other places. (like inside matplotlib, PySide2, numpy, dropbox and such).

My guess is that it's working for most Windows users if Pandas is missing the DLLs because they are found somewhere else. Only some pretty empty installations (like a Docker container or the firewall protected machine I had problems with) will not have those DLLs on the path.

I tried my Python 3.7.4 64bit installation, updated Pandas to 1.0.3 and the "c:\Python37\Lib\site-packages\pandas_libs\window\" directory is also missing the DLLs. But import pandas as pd still works from Python 3.7.4.

[WillAyd]

Is this reproducible outside of Docker? AFAICT msvcp140.dll isn't necessarily a system file but at the same time is required to run C++ applications. There's a similar thread on MSFT forums about it missing:

https://answers.microsoft.com/en-us/windows/forum/windows_10-performance/msvcp140dll-is-missing-in-my-win-10/1c65d6b0-68b8-4b59-b720-3e6a33038389

So maybe the Docker container should just require that instead of changing anything with pandas?

[jaydave1988]

For me, it is working in my laptop, but it is not working in Windows VM.
It is not working even after applying above solution.

[AlexSaliev]

I have the same problem.
Python 3.7.7 32bit
pandas-1.0.3-cp37-cp37m-win32
win 10 64bit

[TomAugspurger]

We have two possible ways to debug this

1. Figure out why pandas 1.0.2 & 1.0.3 are different. This might be changes in the source code from 1.0.1 or changes in the build configuration (Cython version, etc.)
2. Regardless of the reason, figure out a fix (perhaps by changing pandas' setup.py.

I don't think any of the maintainers are on Windows, so if anyone reporting issues is able to debug this we'd appreciate it.

cc @chris-b1 if you have time to look and still have access to a Windows machine.

[chris-b1]

@TomAugspurger - could you point me where the wheel building stuff is? I still do have windows and will take a look

[TomAugspurger]

Thanks.
https://github.com/MacPython/pandas-wheels/blob/master/azure/windows.yml
and
https://github.com/MacPython/pandas-wheels/blob/master/azure-pipelines.yml
has the wheel building stuff.

Logs are at
https://dev.azure.com/pandas-dev/pandas-wheels/_build?definitionId=3&_a=summary
(LMK if you need permissions on azure-pipelines).

On Wed, Apr 1, 2020 at 10:07 AM chris-b1 [e-mail] wrote:

@TomAugspurger https://github.com/TomAugspurger - could you point me
where the wheel building stuff is? I still do have windows and will take a
look

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/32857#issuecomment-607304971,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKAOIWRYU3C37GUVEN76UDRKNKCTANCNFSM4LQGV4PQ
.

[chris-b1]

xref https://github.com/pandas-dev/pandas/pull/21321#issuecomment-395190010

I think I've got this figured out - just need to copy the DLLs noted in that comment in the wheel building process - working on testing/confirming and will submit a PR

[chris-b1]

Hmm, kind of stuck here. Built a wheel with the DLLs packed, see below, but still get that import error. But, if I install the 1.0 wheel, with the same DLLs packed (was that built by you @cgohlke ?), works fine.

# inside Windows docker container

C:\>dir Python\Lib\site-packages\pandas\_libs\window\*.dll
 Volume in drive C has no label.
 Volume Serial Number is 4676-6F6D

 Directory of C:\Python\Lib\site-packages\pandas\_libs\window

04/02/2020  10:45 AM           333,632 concrt140.dll
04/02/2020  10:45 AM           633,152 msvcp140.dll
               2 File(s)        966,784 bytes
               0 Dir(s)  21,033,172,992 bytes free

C:\python
>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 54, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 65, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 131, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4598, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10212, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed: The specified module could not be found.

[TomAugspurger]

1.0.0 - 1.0.2 are from Christoph. 1.0.3 is the first to be built on azure-pipelines.

[cgohlke]

Hmm, kind of stuck here. Built a wheel with the DLLs packed, see below, but still get that import error. But, if I install the 1.0 wheel, with the same DLLs packed (was that built by you @cgohlke ?), works fine.

In case you are using VS 2019, msvcp140.dll now also depends on vcruntime140_1.dll.

[chris-b1]

Wow, that's it - thanks!

[routetehpacketz]

I ran into this issue today on a Windows 2012 server after updating to Python 3.8 and installing pandas 1.0.3. I found concrt140.dll and msvcp140.dll in C:\Windows\System32 and copied them to the \pandas_libs\window folder as specified here to resolve.

[jshridha]

I just installed python3.8 on a completely fresh installation of Windows 10 1909 and ran into this issue. All I had to do was install the lastest x64 version of the visual c redistributable vc_redist from here and my import problem went away.

[ace-e4s]

We ran into the same issue on our production VM (Windows Server 2016 Data Center). Pandas 1.0.1 works fine, but anything later throws a "Could not load DLL" error. We use Python 3.7.6.

[TomAugspurger]

This is fixed in master. I think https://github.com/MacPython/pandas-wheels/pull/82 is the only remaining change needed for our build infrastructure.

I'm not sure that anyone has time to make a release in the near-term, but these wheels are uploaded to https://anaconda.org/scipy-wheels-nightly/pandas nightly.

[hanzigs]

i'm getting for pandas, python 3.7.4 with pandas 1.0.5

Traceback (most recent call last):
  File "C:\app\FlaskAPI.py", line 14, in <module>
    import pandas as pd
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 55, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 60, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 124, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4572, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10349, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed: The specified module could not be found.

may I know the fix?

[TomAugspurger]

@apremgeorge as far as I know this was fixed in 1.0.4, and it shouldb e fixed in 1.0.5 as well.

Can you post the outputs of

1. creating a clean virtualven
2. pip install pandas==1.0.4
3. python -c "import pandas"

and repeat that for 1.0.5?

[hanzigs]

@TomAugspurger Thanks for the reply
Here is the screenshots, created new env

[hanzigs]

pandas 1.0.1 worked for me
Thanks

[jshridha]

@apremgeorge Try out my suggestion above. It has worked for me on several fresh windows/python installs lately where I was seeing the same issue. https://github.com/pandas-dev/pandas/issues/32857#issuecomment-615510956

[cgohlke]

The VC++ runtime DLLs did not make it into the wheels on PyPI. It looks like the DLLs are copied to the pandas source directory after (not before) building the wheels. https://github.com/MacPython/pandas-wheels/blob/master/azure/windows.yml#L71-L74

[TomAugspurger]

Thanks for the pointer. Will see if https://github.com/MacPython/pandas-wheels/pull/90 fixes it.

[TomAugspurger]

Could people here try out the 1.1.0rc0 wheel to see if the problem is fixed? pip install --pre pandas==1.1.0rc0.

[AaryamanCodes]

Hi, try reinstalling python and do pip install pandas it worked for me! (on windows 10)

[TomAugspurger]

Has anyone here been able to try the pre-release wheels for 1.1.0rc0? We're planning to release 1.1.0 final today or tomorrow and it would be nice to verify that this is fixed.

[artkha1]

Has anyone here been able to try the pre-release wheels for 1.1.0rc0? We're planning to release 1.1.0 final today or tomorrow and it would be nice to verify that this is fixed.

Tried pip install --pre pandas==1.1.0rc0, the issue was fixed.

[mborus]

Tried this with Python3.8 64bit & Python3.8 32bit (the 32bit never had pandas installed) and it worked for me.

[flsmidth-thw]

On a docker image with python 3.7.8 and Windows-10-10.0.17763-SP0, we can reproduce the DLL error with pandas==1.0.5, and in a fresh instance get no error with pandas==1.1.0.rc0 🥳

[TomAugspurger]

Thanks for confirming. 1.1.0 will be out later today.