Cryptography: Strange issue I encountered.

[TheOtherGuy]

I am coding a script using Netmiko (utilises Cryptography module for certain tasks) and the re module when I stumbled upon this strange issue.

pattern = re.compile(r'((FG)|(ai))\d/\d{1,2}', re.MULTILINE)
a = pattern.findall(output_errdis)
print(a)
if not a:
     sys.exit()
This lines above will result in the cryptic error below:
extern "Python": function Cryptography_locking_cb() called, but @ffi.def_extern() was not called in the current subinterpreter. Returning 0.

The crucial parts are the 1st and 5th line.

Changing the regex to a pattern that does not involve any (...) containing any character will solve the issue. An empty () without any spacing in between will not result in any error.
Some examples which would lead to the error:
(zz) - Error
( ) - Error
(bb)|(aa) - Error
(&&) - Error

Commenting sys.exit() will solve the problem as well.

I am using Python 3.5.1 64 bit on Windows 10 64 bit.

[reaperhulk]

This is an issue with the way cffi does static callbacks combined with an interaction in shutdown on py 3.5. It's a known issue but very tricky to fix. When we ship a cryptography with OpenSSL 1.1.0 in a month or so we'll be able to bypass this problem though (as OpenSSL does its own locking finally)

In the meantime the error is harmless but annoying, sorry about that!

[collisdigital]

If you're seeing this running inside Apache see (https://cryptography.io/en/latest/faq/#starting-cryptography-using-mod-wsgi-produces-an-internalerror-during-a-call-in-register-osrandom-engine) I say this because of the mention of sub interpreters which made me wonder if your running it this way.

[alex]

Yeah, I'm going to close this, there's nothing that can be done here.

[rgallart101]

If this could be of any help, I've just ran through that problem today. In my case, that exact same message appeared because inadvertently I was opening twice the same ssh connection.

[mhils]

When we ship a cryptography with OpenSSL 1.1.0 in a month or so

@reaperhulk: May I ask what's the plan for Linux here? Are you hoping for distros to catch up or are you going to ship a manylinux wheel?

[reaperhulk]

We haven't decided yet, although a few people at PyCon during sprints made a reasonable case for shipping a manylinux1 wheel.

[TheOtherGuy]

@reaperhulk , is there any way to prevent the output of the error extern "Python.... in Windows console ?

I tried temporarily suppressing stdout and stderr to no avail.

save_stderr = sys.stderr
sys.stderr = open(os.devnull, 'w')
sys.stderr = save_stderr

save_stdout = sys.stdout
sys.stdout = open(os.devnull, 'w')
sys.stdout = save_stdout

[reaperhulk]

@TheOtherGuy not that I'm aware of. We may switch off def_extern as well, which would resolve this (albeit potentially causing us some trouble with SELinux)

[TheOtherGuy]

@reaperhulk , oh darn, thanks. There is a particular condition in my script that causes this happen and I hope to hide the output from the user. Will a future release of cryptography using OpenSSL 1.1.0 resolve this issue ?

[reaperhulk]

Yes, 1.1.0 will resolve this issue, as would a hypothetical release that stops using the static callbacks.

[TheOtherGuy]

I see, thanks :) . May I know if there's any estimated timeline for the release with OpenSSL 1.10 ?

[reaperhulk]

Probably a week or two after 1.1.0 comes out, which doesn't have a date really but should be "soon-ish"

[TheOtherGuy]

I managed to suppress stderr which is printing the extern.. error by modifying the code from a Stackoverflow post.:

def redirect_stderr():
    print "Redirecting stderr"
    sys.stderr.flush()
    newstderr = os.dup(2)
    devnull = os.open(os.devnull, os.O_WRONLY)
    os.dup2(devnull, 2)
    os.close(devnull)
    sys.stderr = os.fdopen(newstderr, 'w')