Pythonnet: Python.Runtime.dll.config under Mac OSX, Anaconda and brew mono

[spichardo]

Hi,

First I installed mono with brew:
brew install mono

With anaconda under OSX El Capitan I managed to install pythonnet with no issues the development version with
pip install --pre pythonnet

Then, when trying to import clr in python I got the error
ImportError: System.DllNotFoundException: libpython2.7.so

I noticed the error was in Python.Runtime.dll.config since it assumes that any non-windows OS will have .so as extension for the libpython library; by example libpython2.7.so , but in anaconda and OSX, this should be libpython2.7.dylib.

This can be easily fixed by being more specific in Python.Runtime.dll.config about the os, such as:

<dllmap dll="python27.dll" target="libpython2.7.dylib" os="osx" />
<dllmap dll="python27" target="libpython2.7.so" os="linux" />

after modifying lib/python2.7/site-packages/Python.Runtime.dll.config using the previous convention then "import clr" works. All my code that uses pythonnet in Windows works with no issues in OSX. I'm attaching the modified Python.Runtime.dll.config. I do not know if other python installations in OSX have a different extension other than .dylib. AFAIR, Canopy uses also dylib, which is the standard for OSX dynamic libraries.

Regards

Sam
Python.Runtime.dll.config.zip

[tonyroberts]

What does the Python that ships as part of OS X use? I remember testing this a while ago and not having any issues (but not using anaconda or canopy, I think I built it myself).

[spichardo]

Hi,

it is .dylib, at least in El Capitan

$ ls -l /usr/lib/libpy*
lrwxr-xr-x 1 root wheel 18 30 Sep 20:27 /usr/lib/libpython.dylib -> libpython2.7.dylib
lrwxr-xr-x 1 root wheel 68 30 Sep 20:27 /usr/lib/libpython2.6.dylib -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/Python
lrwxr-xr-x 1 root wheel 68 30 Sep 20:27 /usr/lib/libpython2.7.dylib -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/Python

Maybe setup.py could generate this file automatically after inquiring the Python linking library,

Regards

S.

[tonyroberts]

Yep, that's a good idea.

[spichardo]

Hi,

I finally had some time to come with a better fix. As discussed, I made some changes in setup.py to generate automatically Python.Runtime.dll.config for Linux and OS X based on 'LDLIBRARY' and 'VERSION' environment variables (get_config_var). I'm attaching the modified setup.py file. You may notice that now Python.Runtime.dll.config is being installed now from the build_lib directory. I also added to copy the generic Python.Runtime.dll.config to the build_lib directory if OS is Windows just for completeness.

There is one little caveat, I do not know if 'LDLIBRARY' is a good source if Python is statically linked, so this setup.py will certainly fail in that case, I do not have a Python installation that is statically linked to test. I just verified it works on OS X and Windows, both using Anaconda. For the moment, for statically linked is copying the generic Python.Runtime.dll.config. I'm attaching the modified setup.py file.
setup.py.zip

This is the git diff output:

diff --git a/setup.py b/setup.py
index b10a19b..95c4549 100644
--- a/setup.py
+++ b/setup.py
@@ -13,6 +13,7 @@ from glob import glob
 import fnmatch
 import sys
 import os
+import shutil

 CONFIG = "Release"  # Release or Debug
 DEVTOOLS = "MsDev" if sys.platform == "win32" else "Mono"
@@ -135,18 +136,28 @@ class PythonNET_BuildExt(build_ext):

         if CONFIG == "Debug":
             defines.extend(["DEBUG", "TRACE"])
-
+        
+        bUseDynamicRuntimeDllConfig = False
+        
         if sys.platform != "win32" and DEVTOOLS == "Mono":
             if sys.platform == "darwin":
                 defines.append("MONO_OSX")
             else:
                 defines.append("MONO_LINUX")
-
             # Check if --enable-shared was set when Python was built
             enable_shared = get_config_var("Py_ENABLE_SHARED")
             if enable_shared == 0:
                 defines.append("PYTHON_WITHOUT_ENABLE_SHARED")
-
+            else:
+                bUseDynamicRuntimeDllConfig=True         
+        if bUseDynamicRuntimeDllConfig:
+             #We will update "Python.Runtime.dll.config"
+             self._update_dll_config(dest_dir)
+        else:
+            #we just copy the generic file for completeness,
+             # data_file below points now "{build_lib}/Python.Runtime.dll.config" for installation
+             shutil.copyfile('Python.Runtime.dll.config',os.path.join(dest_dir,'Python.Runtime.dll.config'))
+            
         if hasattr(sys, "abiflags"):
             if "d" in sys.abiflags:
                 defines.append("PYTHON_WITH_PYDEBUG")
@@ -176,7 +187,38 @@ class PythonNET_BuildExt(build_ext):

         if DEVTOOLS == "Mono":
             self._build_monoclr(ext)
-
+            
+    def _update_dll_config(self,build_dir):
+        PythonLibFname=get_config_var('LDLIBRARY')
+        print "pythonlib =", PythonLibFname
+        if PythonLibFname==None:
+            raise RuntimeError("_update_dll_config: config variable 'LDLIBRARY' not present. Unable to detect file extension of Python library")
+        Version=get_config_var('VERSION')
+        IntVersion = str(int(float(Version)*10))
+        if PythonLibFname==None:
+            raise RuntimeError("_update_dll_config: config variable 'VERSION' not present. Unable to detect Python version")
+        Preamble=['<!-- Mono DLL map for Python.Runtime.dll\n',
+                  '\n',
+                  'Keep this file next to Python.Runtime.dll\n',
+                  '\n',
+                  'For more information read:\n',
+                  '  http://www.mono-project.com/Config\n',
+                  '  http://www.mono-project.com/Config_DllMap\n'
+                  '\n',
+                  '-->\n',
+                  '\n',
+                  '<configuration>\n'];
+        Ending =['</configuration>\n']
+        with file(os.path.abspath(os.path.join(build_dir,"Python.Runtime.dll.config")),'w') as fconfig:
+            for l in Preamble:
+                fconfig.write(l)
+            PythonDllconfig ='    <dllmap dll="python' +IntVersion+'" target ="' + PythonLibFname+'" os="!windows" />\n'
+            fconfig.write(PythonDllconfig)
+            PythonDllconfig ='    <dllmap dll="python' +IntVersion+'.dll" target ="' + PythonLibFname+'" os="!windows" />\n'
+            fconfig.write(PythonDllconfig)
+            for l in Ending:
+                fconfig.write(l)   
+                 
     def _get_manifest(self, build_dir):
         if DEVTOOLS == "MsDev" and sys.version_info[:2] > (2,5):
             mt = _find_msbuild_tool("mt.exe", use_windows_sdk=True)
@@ -318,7 +360,7 @@ if __name__ == "__main__":
         data_files=[
             ("{install_platlib}", [
                 "{build_lib}/Python.Runtime.dll",
-                "Python.Runtime.dll.config"]),
+                "{build_lib}/Python.Runtime.dll.config"]),
         ],
         zip_safe=False,
         cmdclass={

[denfromufa]

@spichardo this patch needs to be tested through conda-forge on osx:

https://github.com/pythonnet/pythonnet/issues/210

[mateuszlewko]

@spichardo I had the same problem and your patch to setup.py solved it! Awesome and thank you 👍

[denfromufa]

@spichardo @mateuszlewko I created a new branch in pythonnet to get Travis CI builds working both on OSX and Linux. If you have any input please let me know. Currently these builds are failing, but I will keep updating that branch:

https://github.com/pythonnet/pythonnet/tree/mac-osx

[vmuriart]

@spichardo was there a downside to just updating the dll.config?

On linux this file is only necessary for PY27. I kept all PY3 version on the file so that the file wasn't so bare.

Looks like this is the case for macOS and PY3 as well, on #392 there was no mention of having to update the dll.config file and the only obvious difference I saw was that it was using PY35. @fractus did you have to update the Python.Runtime.dll.config for pythonnet to work in your case?

[fractus]

@vmuriart -- No, just the changes reported in #392.

[vmuriart]

Fixed the issue by removing the dependency on the dll.config altogether. Let us know if you still have issues.

[denfromufa]

@vmuriart so mono_domain_set_config is not required anymore?

https://github.com/pythonnet/pythonnet/pull/189/files

[vmuriart]

Probably not needed since the file doesn't exist anymore.

[denfromufa]

@vmuriart removing this setting may be now regression for this issue:

https://github.com/pythonnet/pythonnet/issues/180

It will be harder to test this though.