Cellprofiler: Conda Mac installation MySQL problem

[dlogan]

I just got the Conda Mac installation developer version going (thanks for writing these @karhohs) . But I get this mysql error when opening CellProfiler below. I have already activated the 'cellprofiler-master' environment:

$ pythonw CellProfiler.py
MySQL could not be loaded.
Traceback (most recent call last):
  File "/Users/logand05/CellProfiler_conda/src/cellprofiler-master/cellprofiler/modules/exporttodatabase.py", line 75, in <module>
    import MySQLdb
  File "build/bdist.macosx-10.5-x86_64/egg/MySQLdb/__init__.py", line 19, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 7, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so, 2): Library not loaded: libssl.1.0.0.dylib
  Referenced from: /Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so
  Reason: image not found
Pipeline saved with CellProfiler version 300
Pipeline saved with CellProfiler version 300
Pipeline saved with CellProfiler version 300
Pipeline saved with CellProfiler version 300

Is maybe 'mysql-python' the incorrect dependency?

[mcquin]

@dlogan do you use homebrew? can you try installing mysql with brew install mysql?

[0x00b1]

@dlogan Homebrew is one option. conda install mysql-python is another.

[dlogan]

I used the conda install environment settings on the CP Conda wiki which include 'mysql-python'. But just to be sure, I did it manually:

$ conda install mysql-python
Fetching package metadata .......
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at /Users/logand05/anaconda2/envs/cellprofiler-master:
#
mysql-python              1.2.5                    py27_0    goodman
(/Users/logand05/anaconda2/envs/cellprofiler-master) [logand05@AMRL2RK0CSFVH8:~/CellProfiler_conda/src/cellprofiler-master]$ pythonw CellProfiler.py
MySQL could not be loaded.
Traceback (most recent call last):
  File "/Users/logand05/CellProfiler_conda/src/cellprofiler-master/cellprofiler/modules/exporttodatabase.py", line 75, in <module>
    import MySQLdb
  File "build/bdist.macosx-10.5-x86_64/egg/MySQLdb/__init__.py", line 19, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 7, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so, 2): Library not loaded: libssl.1.0.0.dylib
  Referenced from: /Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so
  Reason: image not found
...

Not sure why that didn't help, but @mcquin I do have brew installed, so I will try that next...

[dlogan]

brew install mysql didn't work either, ugh.

$ brew install mysql
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> Updated Formulae
abcl                     conan                    folly                    isl ✔                    meson                    node                     tbb
agda                     consul                   geoserver                kops                     minizinc                 osquery                  terragrunt
arangodb                 curl                     git-annex                kubernetes-cli           monetdb                  rocksdb                  tor
aws-sdk-cpp              epeg                     goenv                    lcov                     mongo-cxx-driver         swiftformat              webp ✔
awscli                   fabric                   h2o                      libhttpseverywhere       mypy                     swiftlint                xonsh
cabal-install            flow                     ice                      libvoikko                nmap                     sysdig                   youtube-dl

Error: mysql-5.7.13 already installed
To install this version, first `brew unlink mysql`

So I unlinked as suggested:

$ brew unlink mysql
Unlinking /usr/local/Cellar/mysql/5.7.13... 93 symlinks removed

Then installed again:

$ brew install mysql
==> Downloading https://homebrew.bintray.com/bottles/mysql-5.7.16.el_capitan.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring mysql-5.7.16.el_capitan.bottle.1.tar.gz
==> Using the sandbox
==> Caveats
We've installed your MySQL database without a root password. To secure it run:
    mysql_secure_installation

To connect run:
    mysql -uroot

To have launchd start mysql now and restart at login:
  brew services start mysql
Or, if you don't want/need a background service you can just run:
  mysql.server start
==> Summary
🍺  /usr/local/Cellar/mysql/5.7.16: 13,511 files, 439M

I didn't do any of the other suggestions above (don't need to, right?), and still get the same issue:

$ pythonw CellProfiler.py
MySQL could not be loaded.
Traceback (most recent call last):
  File "/Users/logand05/CellProfiler_conda/src/cellprofiler-master/cellprofiler/modules/exporttodatabase.py", line 75, in <module>
    import MySQLdb
  File "build/bdist.macosx-10.5-x86_64/egg/MySQLdb/__init__.py", line 19, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 7, in <module>
  File "build/bdist.macosx-10.5-x86_64/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so, 2): Library not loaded: libssl.1.0.0.dylib
  Referenced from: /Users/logand05/Library/Caches/Python-Eggs/MySQL_python-1.2.5-py2.7-macosx-10.5-x86_64.egg-tmp/_mysql.so
  Reason: image not found

[0x00b1]

I misread your error. Try:

brew install --upgrade openssl
brew unlink openssl && brew link --overwrite openssl --force

[dlogan]

This didn't quite do it. It was not linking (BUT SEE FIX BELOW). The latter command in the second line gave this:

$ brew unlink openssl && brew link --overwrite openssl --force
Unlinking /usr/local/Cellar/openssl/1.0.2j... 0 symlinks removed
Warning: Refusing to link: openssl
Linking keg-only openssl means you may end up linking against the insecure,
deprecated system OpenSSL while using the headers from Homebrew's openssl.
Instead, pass the full include/library paths to your compiler e.g.:
  -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib

Googling led me down this rabbit hole: https://github.com/dotnet/cli/issues/3964
which is similar to my issue but I am not doing anything with dotnet...

Another blog post worked though!
https://medium.com/@zlwaterfield/openssl-with-el-capitan-456bf68bf43a#.224eq1k1c
Basically, I needed to do this:

ln -s /usr/local/Cellar/openssl/1.0.2j/include/openssl /usr/local/include/openssl

and now CP runs without issues!

Feel free to close this if you think this linking issue was mine alone, otherwise it would be nice to have this added somewhere.

[0x00b1]

Nice!

@mcquin mentioned she ran into this issue too. I’ll keep this open since it’s worth documenting somewhere.

[mcquin]

Created a troubleshooting page on the wiki with information from this issue, which we can add to over time. Page linked under "Installation > Troubleshooting" on the wiki nav bar.

[gandalf1819]

I'm getting the following error.

(base) Chinmays-MacBook-Pro:elk chinmay$ conda install -c anaconda mysql-python
Collecting package metadata: done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

• mysql-python

Current channels:

• https://conda.anaconda.org/anaconda/osx-64
• https://conda.anaconda.org/anaconda/noarch
• https://repo.anaconda.com/pkgs/main/osx-64
• https://repo.anaconda.com/pkgs/main/noarch
• https://repo.anaconda.com/pkgs/free/osx-64
• https://repo.anaconda.com/pkgs/free/noarch
• https://repo.anaconda.com/pkgs/r/osx-64
• https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.