Dlib: Python process hanging on exit after calling cnn_face_detection_model_v1 (Windows)

[patricksnape]

Build Info

• Visual Studio 2015 on Appveyor
• Building for conda-forge
• Builds are here
  
  
  
  • Hanging builds 1, 2
  
  
• No errors during building
• Boost 1.65.1

CMake Config

cmake ..\tools\python -LAH -G"NMake Makefiles"              ^
-DCMAKE_BUILD_TYPE="Release"                                ^
-DCMAKE_PREFIX_PATH="%LIBRARY_PREFIX%"                      ^
-DBoost_USE_STATIC_LIBS=0                                   ^
-DBoost_USE_STATIC_RUNTIME=0                                ^
-DBOOST_ROOT="%LIBRARY_PREFIX%"                             ^
-DBOOST_INCLUDEDIR="%LIBRARY_INC%"                          ^
-DBOOST_LIBRARYDIR="%LIBRARY_LIB%"                          ^
-DPYTHON3=%PY3K%                                            ^
-DPYTHON_LIBRARY="%PREFIX%\libs\python%PY_VER_NO_DOT%.lib"  ^
-DPYTHON_INCLUDE_DIR="%PREFIX%\include"                     ^
-DDLIB_LINK_WITH_SQLITE3=0                                  ^
-DDLIB_PNG_SUPPORT=1                                        ^
-DPNG_INCLUDE_DIR="%LIBRARY_INC%"                           ^
-DPNG_PNG_INCLUDE_DIR="%LIBRARY_INC%"                       ^
-DPNG_LIBRARY="%PNG_LIBRARY%"                               ^
-DZLIB_INCLUDE_DIRS="%LIBRARY_INC%"                         ^
-DZLIB_LIBRARIES="%LIBRARY_BIN%\zlib.dll"                   ^
-DDLIB_JPEG_SUPPORT=1                                       ^
-DDLIB_USE_BLAS=0                                           ^
-DDLIB_USE_LAPACK=0                                         ^
-DUSE_SSE2_INSTRUCTIONS=1                                   ^
-DUSE_SSE4_INSTRUCTIONS=0                                   ^
-DUSE_AVX_INSTRUCTIONS=0                                    ^
-DDLIB_USE_CUDA=0                                           ^
-DDLIB_GIF_SUPPORT=0

Reproducing snippet

import sys
import dlib
import numpy as np
from PIL import Image
import bz2

MMOD_URL = 'http://dlib.net/files/mmod_human_face_detector.dat.bz2'
# By участница Udacha (личный архив участницы) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons
IMAGE_URL = 'https://upload.wikimedia.org/wikipedia/commons/6/67/--%D0%98%D0%B7%D0%BE%D0%B1%D1%80%D0%B0%D0%B6%D0%B5%D0%BD%D0%B8%D0%B5-%D0%9F%D0%BE%D1%80%D1%82%D1%80%D0%B5%D1%82%D1%8B-%D0%9C%D0%B8%D1%85%D0%B0%D0%B9%D0%BB%D0%BE%D0%B2%D0%B0_%D0%95%D0%BB%D0%B5%D0%BD%D0%B0_%D0%92%D0%BB%D0%B0%D0%B4%D0%B8%D0%BC%D0%B8%D1%80%D0%BE%D0%B2%D0%BD%D0%B0.jpg'

def download_file(url, out_path):
    import requests

    with requests.get(url, stream=True) as r:
        with open(out_path, 'wb') as f:
            for chunk in r.iter_content(1024):
                f.write(chunk)

def bz2_decompress_inplace(path, out_path):
    with open(path, 'rb') as source:
        with open(out_path, 'wb') as dest:
            dest.write(bz2.decompress(source.read()))

print('Downloading CNN Model')
cnn_model_bz2_local_path = './model.dat.bz2'
cnn_model_dat_local_path = './model.dat'
download_file(MMOD_URL, cnn_model_bz2_local_path)
bz2_decompress_inplace(cnn_model_bz2_local_path, cnn_model_dat_local_path)

print('Downloading Face Image')
image_local_path = './face.png'
download_file(IMAGE_URL, image_local_path)

detector = dlib.cnn_face_detection_model_v1(cnn_model_dat_local_path)
image = np.array(Image.open(image_local_path))
print('Perform detection')
results = detector(image)
print('Detection finished - should immediately exit')
sys.exit(0)

Description

For some reason the Python process hangs indefinitely after a CNN detection is performed on Windows. This doesn't happen on OSX or Linux and it doesn't happen if you only build the cnn_face_detection_model_v1 detector - you must execute the detector - which will complete successfully and the Python process continutes as expected. However, when exiting the process in any way (normal termination, exception raised, explicitly calling sys.exit) the process hangs indefinitely.

Looks to me like something funky with the threading keeping the process alive?

[davisking]

Are you sure it's executed the detector? The logs you linked to don't seem to show any print statements after downloading the model files?

[davisking]

Just checked, it's happening. So windows does this asinine thing where when dlls are unloaded it just abruptly terminates threads. It doesn't run destructors before doing this or anything nice like that. So you can get into problems obviously since threads just vanish. In this case, the code that's actually responsible for shutting the threads down is deadlocking because it's trying to do stuff with threads that suddenly don't exist.

I thought I had dealt with all the parts of dlib's code that are impacted by this behavior already :(

I'll look into it.

[patricksnape]

Thanks Davis - sorry to make you work on Windows 🤣

[davisking]

Ha, yeah, I'm not a huge windows fan. But thanks for reporting this issue.

I just pushed a change that seems to fix it. Let me know if your tests work now.

[davisking]

Oops, pushed the wrong changsets. Just pushed the change to github.

[patricksnape]

Build queued: https://ci.appveyor.com/project/conda-forge/dlib-feedstock/build/1.0.38

Thanks for the quick response!

[davisking]

No problem.

[patricksnape]

Build passed: https://ci.appveyor.com/project/conda-forge/dlib-feedstock/build/1.0.38/job/y7o7d1l1b190jabh

Next controversial question - it is possible to publish a release 19.8 so I can distribute the binaries for this 😄 ?

[davisking]

Cool.

Ha, well, I'm going to make 19.8 soon, in like a week I think.

[patricksnape]

FYI I just made a patch from your commit and this also fixed it - so I can temporarily release 19.7 with the patch

[rover886]

@patricksnape Is this issue is still present? I am using python version 3.8.1 along with dlib version 19.19.0. My program does not exit if I perform cnn_faces = cnn_face_detector(image, 0).

[patricksnape]

Sorry I'm not sure - I rarely work on Windows any more! If it's happening again it's likely a new issue and I would open a new issue

[rover886]

@patricksnape, @davisking Yes, its happening again with dlib version 19.19.0. I even tested with program given by dlib http://dlib.net/cnn_face_detector.py.html. Please reopen the issue.

[davisking]

Fill out a new issue and completely fill out the issue template.

[rover886]

Fill out a new issue and completely fill out the issue template.

Created #1983