Opencv-python: Travis Windows: ImportError: DLL load failed: The specified module could not be found.

[pfaion]

Hi, thanks for putting all the work into this! :sparkles:

I have an issue with using opencv-python on Travis Windows instances.
Here a simple import cv2 will result in ImportError: DLL load failed: The specified module could not be found.

The problem is: this even happens with opencv-python-headless.

I prepared a small example: https://travis-ci.org/pfaion/travis-opencv-python-test/builds/614589763#L68-L72

More context

I had similar problems when loading manually compiled opencv DLLs on travis. It appears this stems from travis using a Windows server version without GUI support. For manually compiled opencv DLLs I was able to fix this by excluding all Windows GUI options in cmake for OpenCV:

...
    -DWITH_WIN32UI=OFF\
    -DWITH_DSHOW=OFF\
    -DWITH_MSMF=OFF\
    -DWITH_DIRECTX=OFF\
...

Now I tried to use opencv-python-headless but I still can't load the DLL. I found a thread where other people reported this problem in a travis forum:
https://travis-ci.community/t/python-and-opencv-dll-load-fails-every-time/4431/6

Apparently someone already figured out the cause of the problem here:

opencv-python-headless gives the same result because this is a result of OpenCV-Python being linked with AVFoundation support 1.

So I assume this is just a flag that was not turned off correctly for the headless build? I quickly searched through the issues, but could not found it, so here it is 🙂

[skvark]

Before I'm switching those off I need to know if that has some side effect somewhere. It might be that if those settings are turned off you can't open for example web cams anymore.

[pfaion]

Sounds good, how can we figure this out?

I would have assumed that the headless version is the minimal set of functionality which does not need GUI support. So if opening web cams needs GUI support then maybe this is just not supposed to be supported with headless? Or is this rather a question of documentation? Because currently I also could not find any information about which features are/aren't included in opencv-python-headless.

[skvark]

GUI support refers to opening windows for example with cv2.imshow. Reading video stream from webcam (there's no need to open graphical user interface) requires that there is some backend which supports reading webcams, e.g. cap = cv2.VideoCapture(0 + cv2.CAP_DSHOW). Server environments could be used for example to process images from cameras.

From OpenCV Cmake output:

--   GUI: 
--     Win32 UI:                    YES
--     VTK support:                 NO

--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES (prebuilt binaries)
--       avcodec:                   YES (58.54.100)
--       avformat:                  YES (58.29.100)
--       avutil:                    YES (56.31.100)
--       swscale:                   YES (5.5.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     DirectShow:                  YES
--     Media Foundation:            YES
--       DXVA:                      NO

https://docs.opencv.org/master/d0/da7/videoio_overview.html

DSHOW, MSMF etc. are used for video input / output. It might be that cameras or other devices could be opened only via FFmpeg but I doubt it supports all scenarios on Windows.

On Linux this is quite a lot easier since disabling dependencies under the GUI section removes dependency to libs such as libsm, libxext, libxrender and the headless package works out of the box in server environments. On Windows the situation seems to be a bit trickier since the dependency chain is more complicated. This needs to be investigated further - for example is it enough to disable only DirectX to fix the issue.

[pfaion]

Correct, maybe not all of the flags are needed. I just disabled everything that referenced GUI on Windows and it worked.

I can test for the minimal set of cmake flags to disable for the build on Windows to work. Will come back to you once I have figured that out.

[skvark]

This might be related: https://github.com/skvark/opencv-python/issues/261

[pfaion]

Sadly installing Media Foundation doesn't seem to help: https://travis-ci.org/pfaion/travis-opencv-python-test/jobs/615439058#L20

[pfaion]

TLDR; The WITH_MSMF flag has to be disabled for OpenCV to work on travis.

---

I had 4 flags disabled in my OpenCV build from source:

• WITH_WIN32UI
• WITH_DSHOW
• WITH_MSMF
• WITH_DIRECTX

I verified again that loading the DLL works on Travis Windows when all flags are disabled:
BUILD SUCCESS

When I turned MSMF back on, it did not work anymore: FAIL: MSMF turned back on
Turning any 1 other flag back on would still work:

• SUCCESS: WIN32_UI turned back on
• SUCCESS: DSHOW turned back on
• SUCCESS: DIRECTX turned back on

I then did a final test with only WITH_MSMF=OFF which also were able to load the DLL successfully:
SUCCESS: only MSMF disabled

[Borda]

so can it be disabled also in the pip wheels?

[pfaion]

No unfortunately the wheels do not offer that option. You need to compile OpenCV on your own with the compiler flag WITH_MSMF=OFF

[Borda]

That is not very convenient to do compile every time if you want to test other project, I saw that your build takes about 20min

[pfaion]

You can use travis caches to keep a once-built OpenCV between jobs: https://docs.travis-ci.com/user/caching/
Maybe that works for you.

[abhiTronix]

@pfaion Not possible. Jobs in a build matrix are independent, and they do not share storage plus they are not guaranteed to be running simultaneously. See https://github.com/travis-ci/travis-ci/issues/6054#issuecomment-219908393

[Borda]

He meant between consecutive jobs for each configuration over particular builds...

[abhiTronix]

@Borda yes, i meant for the same.

[pfaion]

@abhiTronix it's a new feature from Travis. Your linked comment is very old from 2016. Have a look at caches and workspaces, you can share data between consequtive jobs and between parallel jobs now!

[pfaion]

I experimented with a setup once where I had a pre stage that would be responsible for managing the OpenCV build. It would use caches to store the build across entire travis builds and rebuild in case the cache was not there or I manually triggered a rebuild. The OpenCV build would be made available within jobs via travis workspaces. This all worked very fine. I ended up not using this because of some other non-OpenCV related issues with the build pipeline.

[skvark]

I disabled MSMF from the headless builds. Change will be included in next release.

[pfaion]

This is awesome, thank you so much!

[mshabunin]

FYI, we have added plug-in mode support for Media Foundation backend in OpenCV: https://github.com/opencv/opencv/pull/17009 It means that MSMF will become optional dependency and it will be possible to create single binary for platforms with and without Media Foundation. However this feature did not get into 4.3.0 release and have not been thoroughly tested yet.